2023-07-12 15:09:54,692 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-07-12 15:09:54,871 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2 2023-07-12 15:09:54,883 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-12 15:09:54,884 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7, deleteOnExit=true 2023-07-12 15:09:54,884 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-12 15:09:54,885 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/test.cache.data in system properties and HBase conf 2023-07-12 15:09:54,885 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/hadoop.tmp.dir in system properties and HBase conf 2023-07-12 15:09:54,886 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/hadoop.log.dir in system properties and HBase conf 2023-07-12 15:09:54,886 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-12 15:09:54,887 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-12 15:09:54,887 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-12 15:09:54,980 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-12 15:09:55,281 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-12 15:09:55,285 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-12 15:09:55,285 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-12 15:09:55,286 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-12 15:09:55,286 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 15:09:55,286 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-12 15:09:55,286 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-12 15:09:55,287 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-12 15:09:55,287 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 15:09:55,287 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-12 15:09:55,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/nfs.dump.dir in system properties and HBase conf 2023-07-12 15:09:55,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/java.io.tmpdir in system properties and HBase conf 2023-07-12 15:09:55,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-12 15:09:55,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-12 15:09:55,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-12 15:09:55,679 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 15:09:55,683 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 15:09:55,894 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-12 15:09:56,028 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-12 15:09:56,042 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:09:56,075 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:09:56,101 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/java.io.tmpdir/Jetty_localhost_localdomain_40803_hdfs____2uv8d5/webapp 2023-07-12 15:09:56,229 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:40803 2023-07-12 15:09:56,237 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-12 15:09:56,237 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-12 15:09:56,579 WARN [Listener at localhost.localdomain/34607] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:09:56,636 WARN [Listener at localhost.localdomain/34607] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:09:56,652 WARN [Listener at localhost.localdomain/34607] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:09:56,657 INFO [Listener at localhost.localdomain/34607] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:09:56,661 INFO [Listener at localhost.localdomain/34607] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/java.io.tmpdir/Jetty_localhost_45541_datanode____.dqria0/webapp 2023-07-12 15:09:56,736 INFO [Listener at localhost.localdomain/34607] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45541 2023-07-12 15:09:56,996 WARN [Listener at localhost.localdomain/35047] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:09:57,008 WARN [Listener at localhost.localdomain/35047] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:09:57,012 WARN [Listener at localhost.localdomain/35047] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:09:57,014 INFO [Listener at localhost.localdomain/35047] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:09:57,023 INFO [Listener at localhost.localdomain/35047] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/java.io.tmpdir/Jetty_localhost_43905_datanode____.p5ldve/webapp 2023-07-12 15:09:57,108 INFO [Listener at localhost.localdomain/35047] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43905 2023-07-12 15:09:57,118 WARN [Listener at localhost.localdomain/38533] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:09:57,133 WARN [Listener at localhost.localdomain/38533] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-12 15:09:57,138 WARN [Listener at localhost.localdomain/38533] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-12 15:09:57,140 INFO [Listener at localhost.localdomain/38533] log.Slf4jLog(67): jetty-6.1.26 2023-07-12 15:09:57,145 INFO [Listener at localhost.localdomain/38533] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/java.io.tmpdir/Jetty_localhost_34351_datanode____.evj2ff/webapp 2023-07-12 15:09:57,234 INFO [Listener at localhost.localdomain/38533] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34351 2023-07-12 15:09:57,242 WARN [Listener at localhost.localdomain/46471] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-12 15:09:57,420 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc742a8694dc7242: Processing first storage report for DS-fddbce96-2ab2-404a-be5a-da73b0545d48 from datanode d6c01eab-2c69-419b-b4a6-953858ab157f 2023-07-12 15:09:57,421 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc742a8694dc7242: from storage DS-fddbce96-2ab2-404a-be5a-da73b0545d48 node DatanodeRegistration(127.0.0.1:39865, datanodeUuid=d6c01eab-2c69-419b-b4a6-953858ab157f, infoPort=35247, infoSecurePort=0, ipcPort=38533, storageInfo=lv=-57;cid=testClusterID;nsid=2027210301;c=1689174595739), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x23a7a5e612102c7: Processing first storage report for DS-adbc8404-9724-4475-b146-6fa20c9116f4 from datanode 5606ee4f-c045-409e-a742-091e025e583c 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x23a7a5e612102c7: from storage DS-adbc8404-9724-4475-b146-6fa20c9116f4 node DatanodeRegistration(127.0.0.1:43821, datanodeUuid=5606ee4f-c045-409e-a742-091e025e583c, infoPort=42267, infoSecurePort=0, ipcPort=35047, storageInfo=lv=-57;cid=testClusterID;nsid=2027210301;c=1689174595739), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf90fa0b4fba88864: Processing first storage report for DS-4cb81aa0-8313-437f-829f-f0b10ed99df7 from datanode 631c6a2a-c60e-45ea-8ae8-e13efaf3ce20 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf90fa0b4fba88864: from storage DS-4cb81aa0-8313-437f-829f-f0b10ed99df7 node DatanodeRegistration(127.0.0.1:37535, datanodeUuid=631c6a2a-c60e-45ea-8ae8-e13efaf3ce20, infoPort=42117, infoSecurePort=0, ipcPort=46471, storageInfo=lv=-57;cid=testClusterID;nsid=2027210301;c=1689174595739), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc742a8694dc7242: Processing first storage report for DS-88c5825e-9723-4c64-87a4-6f692f545a5e from datanode d6c01eab-2c69-419b-b4a6-953858ab157f 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc742a8694dc7242: from storage DS-88c5825e-9723-4c64-87a4-6f692f545a5e node DatanodeRegistration(127.0.0.1:39865, datanodeUuid=d6c01eab-2c69-419b-b4a6-953858ab157f, infoPort=35247, infoSecurePort=0, ipcPort=38533, storageInfo=lv=-57;cid=testClusterID;nsid=2027210301;c=1689174595739), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:09:57,422 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x23a7a5e612102c7: Processing first storage report for DS-595c9cd7-8d04-4d83-bc85-c1ca022d55e9 from datanode 5606ee4f-c045-409e-a742-091e025e583c 2023-07-12 15:09:57,423 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x23a7a5e612102c7: from storage DS-595c9cd7-8d04-4d83-bc85-c1ca022d55e9 node DatanodeRegistration(127.0.0.1:43821, datanodeUuid=5606ee4f-c045-409e-a742-091e025e583c, infoPort=42267, infoSecurePort=0, ipcPort=35047, storageInfo=lv=-57;cid=testClusterID;nsid=2027210301;c=1689174595739), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-12 15:09:57,423 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf90fa0b4fba88864: Processing first storage report for DS-b12c0426-3d41-4eec-9aea-9b84b579fcdb from datanode 631c6a2a-c60e-45ea-8ae8-e13efaf3ce20 2023-07-12 15:09:57,423 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf90fa0b4fba88864: from storage DS-b12c0426-3d41-4eec-9aea-9b84b579fcdb node DatanodeRegistration(127.0.0.1:37535, datanodeUuid=631c6a2a-c60e-45ea-8ae8-e13efaf3ce20, infoPort=42117, infoSecurePort=0, ipcPort=46471, storageInfo=lv=-57;cid=testClusterID;nsid=2027210301;c=1689174595739), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-12 15:09:57,574 DEBUG [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2 2023-07-12 15:09:57,625 INFO [Listener at localhost.localdomain/46471] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/zookeeper_0, clientPort=63723, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-12 15:09:57,636 INFO [Listener at localhost.localdomain/46471] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63723 2023-07-12 15:09:57,649 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:57,652 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:58,273 INFO [Listener at localhost.localdomain/46471] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7 with version=8 2023-07-12 15:09:58,273 INFO [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/hbase-staging 2023-07-12 15:09:58,505 INFO [Listener at localhost.localdomain/46471] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-12 15:09:58,859 INFO [Listener at localhost.localdomain/46471] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 15:09:58,883 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:58,883 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:58,883 INFO [Listener at localhost.localdomain/46471] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:09:58,883 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:58,884 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:09:58,993 INFO [Listener at localhost.localdomain/46471] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:09:59,049 DEBUG [Listener at localhost.localdomain/46471] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-12 15:09:59,127 INFO [Listener at localhost.localdomain/46471] ipc.NettyRpcServer(120): Bind to /148.251.75.209:37639 2023-07-12 15:09:59,137 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,139 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,158 INFO [Listener at localhost.localdomain/46471] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37639 connecting to ZooKeeper ensemble=127.0.0.1:63723 2023-07-12 15:09:59,190 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:376390x0, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:09:59,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37639-0x10081bb5c6c0000 connected 2023-07-12 15:09:59,221 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 15:09:59,221 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:09:59,225 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 15:09:59,234 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37639 2023-07-12 15:09:59,234 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37639 2023-07-12 15:09:59,235 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37639 2023-07-12 15:09:59,235 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37639 2023-07-12 15:09:59,236 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37639 2023-07-12 15:09:59,242 INFO [Listener at localhost.localdomain/46471] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7, hbase.cluster.distributed=false 2023-07-12 15:09:59,302 INFO [Listener at localhost.localdomain/46471] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 15:09:59,303 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,303 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,303 INFO [Listener at localhost.localdomain/46471] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:09:59,303 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,303 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:09:59,307 INFO [Listener at localhost.localdomain/46471] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:09:59,310 INFO [Listener at localhost.localdomain/46471] ipc.NettyRpcServer(120): Bind to /148.251.75.209:41741 2023-07-12 15:09:59,312 INFO [Listener at localhost.localdomain/46471] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:09:59,317 DEBUG [Listener at localhost.localdomain/46471] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:09:59,319 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,320 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,323 INFO [Listener at localhost.localdomain/46471] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41741 connecting to ZooKeeper ensemble=127.0.0.1:63723 2023-07-12 15:09:59,328 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:417410x0, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:09:59,330 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41741-0x10081bb5c6c0001 connected 2023-07-12 15:09:59,330 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 15:09:59,331 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:09:59,332 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 15:09:59,332 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41741 2023-07-12 15:09:59,333 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41741 2023-07-12 15:09:59,333 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41741 2023-07-12 15:09:59,334 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41741 2023-07-12 15:09:59,334 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41741 2023-07-12 15:09:59,346 INFO [Listener at localhost.localdomain/46471] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 15:09:59,346 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,346 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,347 INFO [Listener at localhost.localdomain/46471] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:09:59,347 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,347 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:09:59,347 INFO [Listener at localhost.localdomain/46471] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:09:59,349 INFO [Listener at localhost.localdomain/46471] ipc.NettyRpcServer(120): Bind to /148.251.75.209:33703 2023-07-12 15:09:59,350 INFO [Listener at localhost.localdomain/46471] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:09:59,351 DEBUG [Listener at localhost.localdomain/46471] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:09:59,352 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,355 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,358 INFO [Listener at localhost.localdomain/46471] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33703 connecting to ZooKeeper ensemble=127.0.0.1:63723 2023-07-12 15:09:59,361 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:337030x0, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:09:59,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33703-0x10081bb5c6c0002 connected 2023-07-12 15:09:59,363 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 15:09:59,364 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:09:59,365 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 15:09:59,365 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33703 2023-07-12 15:09:59,365 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33703 2023-07-12 15:09:59,366 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33703 2023-07-12 15:09:59,366 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33703 2023-07-12 15:09:59,367 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33703 2023-07-12 15:09:59,378 INFO [Listener at localhost.localdomain/46471] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-12 15:09:59,378 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,378 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,378 INFO [Listener at localhost.localdomain/46471] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-12 15:09:59,378 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-12 15:09:59,379 INFO [Listener at localhost.localdomain/46471] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-12 15:09:59,379 INFO [Listener at localhost.localdomain/46471] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-12 15:09:59,380 INFO [Listener at localhost.localdomain/46471] ipc.NettyRpcServer(120): Bind to /148.251.75.209:37993 2023-07-12 15:09:59,380 INFO [Listener at localhost.localdomain/46471] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-12 15:09:59,381 DEBUG [Listener at localhost.localdomain/46471] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-12 15:09:59,382 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,383 INFO [Listener at localhost.localdomain/46471] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,384 INFO [Listener at localhost.localdomain/46471] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37993 connecting to ZooKeeper ensemble=127.0.0.1:63723 2023-07-12 15:09:59,389 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:379930x0, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-12 15:09:59,390 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:379930x0, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 15:09:59,390 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37993-0x10081bb5c6c0003 connected 2023-07-12 15:09:59,391 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:09:59,391 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-12 15:09:59,392 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37993 2023-07-12 15:09:59,393 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37993 2023-07-12 15:09:59,393 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37993 2023-07-12 15:09:59,394 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37993 2023-07-12 15:09:59,394 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37993 2023-07-12 15:09:59,395 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:09:59,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-12 15:09:59,407 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:09:59,423 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 15:09:59,423 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 15:09:59,423 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 15:09:59,423 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-12 15:09:59,424 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:09:59,426 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-12 15:09:59,427 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,37639,1689174598377 from backup master directory 2023-07-12 15:09:59,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-12 15:09:59,430 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:09:59,430 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-12 15:09:59,431 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:09:59,431 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:09:59,434 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-12 15:09:59,435 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-12 15:09:59,522 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/hbase.id with ID: 17492a16-7ef3-409d-9c83-dea7e7575e39 2023-07-12 15:09:59,574 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-12 15:09:59,593 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:09:59,644 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5fb4a008 to 127.0.0.1:63723 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 15:09:59,675 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@699cb06e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:09:59,694 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:09:59,696 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-12 15:09:59,710 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-12 15:09:59,710 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-12 15:09:59,712 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:09:59,715 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:09:59,716 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:09:59,744 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store-tmp 2023-07-12 15:09:59,816 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:09:59,817 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:09:59,817 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:09:59,817 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:09:59,817 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:09:59,817 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:09:59,817 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:09:59,817 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:09:59,819 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/WALs/jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:09:59,838 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37639%2C1689174598377, suffix=, logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/WALs/jenkins-hbase20.apache.org,37639,1689174598377, archiveDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/oldWALs, maxLogs=10 2023-07-12 15:09:59,889 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK] 2023-07-12 15:09:59,889 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK] 2023-07-12 15:09:59,889 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK] 2023-07-12 15:09:59,897 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:09:59,956 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/WALs/jenkins-hbase20.apache.org,37639,1689174598377/jenkins-hbase20.apache.org%2C37639%2C1689174598377.1689174599847 2023-07-12 15:09:59,956 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK], DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK], DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK]] 2023-07-12 15:09:59,957 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:09:59,957 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:09:59,960 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:09:59,961 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:10:00,011 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:10:00,017 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-12 15:10:00,037 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-12 15:10:00,046 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:00,052 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:10:00,054 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:10:00,069 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-12 15:10:00,074 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:00,075 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71667632, jitterRate=0.06793093681335449}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:10:00,075 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:10:00,076 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-12 15:10:00,097 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:00,098 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:00,100 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:00,102 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:00,127 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 25 msec 2023-07-12 15:10:00,127 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-12 15:10:00,147 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-12 15:10:00,152 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-12 15:10:00,172 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-12 15:10:00,175 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-12 15:10:00,176 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-12 15:10:00,180 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-12 15:10:00,184 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-12 15:10:00,186 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:00,188 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-12 15:10:00,188 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-12 15:10:00,199 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-12 15:10:00,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 15:10:00,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 15:10:00,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 15:10:00,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-12 15:10:00,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:00,206 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase20.apache.org,37639,1689174598377, sessionid=0x10081bb5c6c0000, setting cluster-up flag (Was=false) 2023-07-12 15:10:00,220 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:00,224 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-12 15:10:00,225 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:10:00,228 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:00,232 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-12 15:10:00,233 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:10:00,235 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.hbase-snapshot/.tmp 2023-07-12 15:10:00,297 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(951): ClusterId : 17492a16-7ef3-409d-9c83-dea7e7575e39 2023-07-12 15:10:00,297 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(951): ClusterId : 17492a16-7ef3-409d-9c83-dea7e7575e39 2023-07-12 15:10:00,297 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(951): ClusterId : 17492a16-7ef3-409d-9c83-dea7e7575e39 2023-07-12 15:10:00,303 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:10:00,303 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:10:00,303 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-12 15:10:00,307 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:10:00,307 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:10:00,307 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-12 15:10:00,308 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:10:00,308 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:10:00,308 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-12 15:10:00,310 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:10:00,310 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:10:00,310 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-12 15:10:00,312 DEBUG [RS:1;jenkins-hbase20:33703] zookeeper.ReadOnlyZKClient(139): Connect 0x41445b67 to 127.0.0.1:63723 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 15:10:00,313 DEBUG [RS:2;jenkins-hbase20:37993] zookeeper.ReadOnlyZKClient(139): Connect 0x2d90a6b4 to 127.0.0.1:63723 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 15:10:00,313 DEBUG [RS:0;jenkins-hbase20:41741] zookeeper.ReadOnlyZKClient(139): Connect 0x3c27caab to 127.0.0.1:63723 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 15:10:00,326 DEBUG [RS:1;jenkins-hbase20:33703] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f83ad3f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:10:00,326 DEBUG [RS:0;jenkins-hbase20:41741] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32e0335f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:10:00,327 DEBUG [RS:2;jenkins-hbase20:37993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23574d76, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:10:00,327 DEBUG [RS:1;jenkins-hbase20:33703] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4bbc3017, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 15:10:00,327 DEBUG [RS:0;jenkins-hbase20:41741] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36dfa1db, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 15:10:00,327 DEBUG [RS:2;jenkins-hbase20:37993] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2768cdc4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 15:10:00,345 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:00,349 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:33703 2023-07-12 15:10:00,351 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:41741 2023-07-12 15:10:00,352 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:37993 2023-07-12 15:10:00,353 INFO [RS:2;jenkins-hbase20:37993] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:10:00,353 INFO [RS:1;jenkins-hbase20:33703] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:10:00,353 INFO [RS:1;jenkins-hbase20:33703] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:10:00,353 INFO [RS:0;jenkins-hbase20:41741] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-12 15:10:00,354 INFO [RS:0;jenkins-hbase20:41741] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:10:00,354 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:10:00,354 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:10:00,353 INFO [RS:2;jenkins-hbase20:37993] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-12 15:10:00,354 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1022): About to register with Master. 2023-07-12 15:10:00,357 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,37639,1689174598377 with isa=jenkins-hbase20.apache.org/148.251.75.209:41741, startcode=1689174599302 2023-07-12 15:10:00,357 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,37639,1689174598377 with isa=jenkins-hbase20.apache.org/148.251.75.209:37993, startcode=1689174599377 2023-07-12 15:10:00,357 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,37639,1689174598377 with isa=jenkins-hbase20.apache.org/148.251.75.209:33703, startcode=1689174599345 2023-07-12 15:10:00,357 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:10:00,358 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:10:00,358 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:10:00,358 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-12 15:10:00,358 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-07-12 15:10:00,358 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,358 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:10:00,359 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,360 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174630360 2023-07-12 15:10:00,363 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-12 15:10:00,369 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:00,369 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-12 15:10:00,375 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 15:10:00,376 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-12 15:10:00,379 DEBUG [RS:2;jenkins-hbase20:37993] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:10:00,381 DEBUG [RS:1;jenkins-hbase20:33703] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:10:00,379 DEBUG [RS:0;jenkins-hbase20:41741] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-12 15:10:00,383 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-12 15:10:00,383 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-12 15:10:00,384 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-12 15:10:00,384 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-12 15:10:00,385 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,387 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-12 15:10:00,388 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-12 15:10:00,389 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-12 15:10:00,392 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-12 15:10:00,393 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-12 15:10:00,394 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689174600394,5,FailOnTimeoutGroup] 2023-07-12 15:10:00,396 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689174600394,5,FailOnTimeoutGroup] 2023-07-12 15:10:00,396 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,396 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-12 15:10:00,398 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,398 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,430 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:00,431 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:00,432 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7 2023-07-12 15:10:00,444 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:55437, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:10:00,444 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50203, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:10:00,444 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:55589, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-12 15:10:00,464 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37639] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,464 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:00,468 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:10:00,472 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/info 2023-07-12 15:10:00,472 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:10:00,473 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:00,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:10:00,476 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:10:00,477 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:10:00,478 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37639] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,480 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37639] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,480 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:00,481 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:10:00,483 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/table 2023-07-12 15:10:00,484 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7 2023-07-12 15:10:00,484 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7 2023-07-12 15:10:00,484 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34607 2023-07-12 15:10:00,484 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:10:00,484 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7 2023-07-12 15:10:00,484 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34607 2023-07-12 15:10:00,485 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:10:00,484 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34607 2023-07-12 15:10:00,485 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-12 15:10:00,485 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:10:00,486 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:00,487 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740 2023-07-12 15:10:00,488 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740 2023-07-12 15:10:00,490 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:10:00,492 DEBUG [RS:1;jenkins-hbase20:33703] zookeeper.ZKUtil(162): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,492 WARN [RS:1;jenkins-hbase20:33703] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:10:00,492 DEBUG [RS:2;jenkins-hbase20:37993] zookeeper.ZKUtil(162): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,492 INFO [RS:1;jenkins-hbase20:33703] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:10:00,492 DEBUG [RS:0;jenkins-hbase20:41741] zookeeper.ZKUtil(162): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,493 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,493 WARN [RS:0;jenkins-hbase20:41741] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:10:00,492 WARN [RS:2;jenkins-hbase20:37993] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-12 15:10:00,494 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,33703,1689174599345] 2023-07-12 15:10:00,494 INFO [RS:2;jenkins-hbase20:37993] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:10:00,494 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,493 INFO [RS:0;jenkins-hbase20:41741] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:10:00,494 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,37993,1689174599377] 2023-07-12 15:10:00,495 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,41741,1689174599302] 2023-07-12 15:10:00,495 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,495 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:10:00,507 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:10:00,512 DEBUG [RS:0;jenkins-hbase20:41741] zookeeper.ZKUtil(162): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,512 DEBUG [RS:2;jenkins-hbase20:37993] zookeeper.ZKUtil(162): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,512 DEBUG [RS:1;jenkins-hbase20:33703] zookeeper.ZKUtil(162): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,512 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:00,512 DEBUG [RS:0;jenkins-hbase20:41741] zookeeper.ZKUtil(162): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,513 DEBUG [RS:2;jenkins-hbase20:37993] zookeeper.ZKUtil(162): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,513 DEBUG [RS:1;jenkins-hbase20:33703] zookeeper.ZKUtil(162): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,513 DEBUG [RS:0;jenkins-hbase20:41741] zookeeper.ZKUtil(162): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,513 DEBUG [RS:2;jenkins-hbase20:37993] zookeeper.ZKUtil(162): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,513 DEBUG [RS:1;jenkins-hbase20:33703] zookeeper.ZKUtil(162): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,513 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63827767, jitterRate=-0.04889215528964996}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:10:00,514 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:10:00,514 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:10:00,514 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:10:00,514 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:10:00,514 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:10:00,514 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:10:00,516 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:10:00,516 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:10:00,521 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:00,521 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-12 15:10:00,525 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-12 15:10:00,525 DEBUG [RS:0;jenkins-hbase20:41741] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-12 15:10:00,525 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-12 15:10:00,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-12 15:10:00,534 INFO [RS:2;jenkins-hbase20:37993] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:10:00,534 INFO [RS:0;jenkins-hbase20:41741] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:10:00,534 INFO [RS:1;jenkins-hbase20:33703] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-12 15:10:00,542 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-12 15:10:00,544 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-12 15:10:00,555 INFO [RS:1;jenkins-hbase20:33703] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:10:00,555 INFO [RS:2;jenkins-hbase20:37993] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:10:00,555 INFO [RS:0;jenkins-hbase20:41741] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-12 15:10:00,559 INFO [RS:1;jenkins-hbase20:33703] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:10:00,559 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,559 INFO [RS:2;jenkins-hbase20:37993] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:10:00,560 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:10:00,560 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,560 INFO [RS:0;jenkins-hbase20:41741] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-12 15:10:00,560 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:10:00,560 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,561 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-12 15:10:00,569 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,569 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,569 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,569 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,569 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:10:00,570 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:10:00,570 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-12 15:10:00,571 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,570 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:2;jenkins-hbase20:37993] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,571 DEBUG [RS:1;jenkins-hbase20:33703] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,572 DEBUG [RS:0;jenkins-hbase20:41741] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-12 15:10:00,575 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,575 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,576 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,588 INFO [RS:1;jenkins-hbase20:33703] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:10:00,590 INFO [RS:2;jenkins-hbase20:37993] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:10:00,591 INFO [RS:0;jenkins-hbase20:41741] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-12 15:10:00,591 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,33703,1689174599345-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,591 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41741,1689174599302-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,591 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37993,1689174599377-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:00,606 INFO [RS:1;jenkins-hbase20:33703] regionserver.Replication(203): jenkins-hbase20.apache.org,33703,1689174599345 started 2023-07-12 15:10:00,606 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,33703,1689174599345, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:33703, sessionid=0x10081bb5c6c0002 2023-07-12 15:10:00,606 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:10:00,606 DEBUG [RS:1;jenkins-hbase20:33703] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,606 DEBUG [RS:1;jenkins-hbase20:33703] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,33703,1689174599345' 2023-07-12 15:10:00,607 DEBUG [RS:1;jenkins-hbase20:33703] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:00,607 DEBUG [RS:1;jenkins-hbase20:33703] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:00,608 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:10:00,608 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:10:00,608 DEBUG [RS:1;jenkins-hbase20:33703] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,608 DEBUG [RS:1;jenkins-hbase20:33703] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,33703,1689174599345' 2023-07-12 15:10:00,608 DEBUG [RS:1;jenkins-hbase20:33703] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-12 15:10:00,609 DEBUG [RS:1;jenkins-hbase20:33703] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-12 15:10:00,609 DEBUG [RS:1;jenkins-hbase20:33703] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:10:00,609 INFO [RS:1;jenkins-hbase20:33703] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:10:00,609 INFO [RS:1;jenkins-hbase20:33703] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:10:00,610 INFO [RS:2;jenkins-hbase20:37993] regionserver.Replication(203): jenkins-hbase20.apache.org,37993,1689174599377 started 2023-07-12 15:10:00,610 INFO [RS:0;jenkins-hbase20:41741] regionserver.Replication(203): jenkins-hbase20.apache.org,41741,1689174599302 started 2023-07-12 15:10:00,610 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,37993,1689174599377, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:37993, sessionid=0x10081bb5c6c0003 2023-07-12 15:10:00,610 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,41741,1689174599302, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:41741, sessionid=0x10081bb5c6c0001 2023-07-12 15:10:00,610 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:10:00,610 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-12 15:10:00,611 DEBUG [RS:0;jenkins-hbase20:41741] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,610 DEBUG [RS:2;jenkins-hbase20:37993] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,611 DEBUG [RS:0;jenkins-hbase20:41741] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,41741,1689174599302' 2023-07-12 15:10:00,612 DEBUG [RS:2;jenkins-hbase20:37993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37993,1689174599377' 2023-07-12 15:10:00,612 DEBUG [RS:2;jenkins-hbase20:37993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:00,612 DEBUG [RS:0;jenkins-hbase20:41741] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:00,612 DEBUG [RS:0;jenkins-hbase20:41741] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:00,612 DEBUG [RS:2;jenkins-hbase20:37993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:00,613 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:10:00,613 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-12 15:10:00,613 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:10:00,613 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-12 15:10:00,613 DEBUG [RS:0;jenkins-hbase20:41741] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:00,614 DEBUG [RS:0;jenkins-hbase20:41741] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,41741,1689174599302' 2023-07-12 15:10:00,614 DEBUG [RS:0;jenkins-hbase20:41741] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-12 15:10:00,614 DEBUG [RS:2;jenkins-hbase20:37993] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:00,614 DEBUG [RS:2;jenkins-hbase20:37993] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37993,1689174599377' 2023-07-12 15:10:00,614 DEBUG [RS:2;jenkins-hbase20:37993] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-12 15:10:00,614 DEBUG [RS:0;jenkins-hbase20:41741] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-12 15:10:00,614 DEBUG [RS:2;jenkins-hbase20:37993] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-12 15:10:00,615 DEBUG [RS:0;jenkins-hbase20:41741] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:10:00,615 INFO [RS:0;jenkins-hbase20:41741] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:10:00,615 INFO [RS:0;jenkins-hbase20:41741] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:10:00,616 DEBUG [RS:2;jenkins-hbase20:37993] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-12 15:10:00,616 INFO [RS:2;jenkins-hbase20:37993] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-12 15:10:00,616 INFO [RS:2;jenkins-hbase20:37993] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-12 15:10:00,698 DEBUG [jenkins-hbase20:37639] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-12 15:10:00,704 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:00,710 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:00,710 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:00,710 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:00,710 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:00,713 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,33703,1689174599345, state=OPENING 2023-07-12 15:10:00,719 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-12 15:10:00,720 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:00,721 INFO [RS:2;jenkins-hbase20:37993] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37993%2C1689174599377, suffix=, logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,37993,1689174599377, archiveDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs, maxLogs=32 2023-07-12 15:10:00,723 INFO [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C33703%2C1689174599345, suffix=, logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,33703,1689174599345, archiveDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs, maxLogs=32 2023-07-12 15:10:00,721 INFO [RS:0;jenkins-hbase20:41741] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C41741%2C1689174599302, suffix=, logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,41741,1689174599302, archiveDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs, maxLogs=32 2023-07-12 15:10:00,723 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-12 15:10:00,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:00,748 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK] 2023-07-12 15:10:00,749 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK] 2023-07-12 15:10:00,749 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK] 2023-07-12 15:10:00,750 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK] 2023-07-12 15:10:00,750 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK] 2023-07-12 15:10:00,750 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK] 2023-07-12 15:10:00,764 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK] 2023-07-12 15:10:00,764 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK] 2023-07-12 15:10:00,764 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK] 2023-07-12 15:10:00,766 INFO [RS:2;jenkins-hbase20:37993] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,37993,1689174599377/jenkins-hbase20.apache.org%2C37993%2C1689174599377.1689174600727 2023-07-12 15:10:00,766 INFO [RS:0;jenkins-hbase20:41741] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,41741,1689174599302/jenkins-hbase20.apache.org%2C41741%2C1689174599302.1689174600728 2023-07-12 15:10:00,767 DEBUG [RS:0;jenkins-hbase20:41741] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK], DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK], DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK]] 2023-07-12 15:10:00,767 DEBUG [RS:2;jenkins-hbase20:37993] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK], DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK], DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK]] 2023-07-12 15:10:00,773 INFO [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,33703,1689174599345/jenkins-hbase20.apache.org%2C33703%2C1689174599345.1689174600727 2023-07-12 15:10:00,774 DEBUG [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK], DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK], DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK]] 2023-07-12 15:10:00,922 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:00,924 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:10:00,928 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46798, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:10:00,939 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-12 15:10:00,939 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-12 15:10:00,944 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C33703%2C1689174599345.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,33703,1689174599345, archiveDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs, maxLogs=32 2023-07-12 15:10:00,969 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK] 2023-07-12 15:10:00,970 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK] 2023-07-12 15:10:00,970 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK] 2023-07-12 15:10:00,980 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/WALs/jenkins-hbase20.apache.org,33703,1689174599345/jenkins-hbase20.apache.org%2C33703%2C1689174599345.meta.1689174600946.meta 2023-07-12 15:10:00,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37535,DS-4cb81aa0-8313-437f-829f-f0b10ed99df7,DISK], DatanodeInfoWithStorage[127.0.0.1:43821,DS-adbc8404-9724-4475-b146-6fa20c9116f4,DISK], DatanodeInfoWithStorage[127.0.0.1:39865,DS-fddbce96-2ab2-404a-be5a-da73b0545d48,DISK]] 2023-07-12 15:10:00,981 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:00,986 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:00,986 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-12 15:10:01,002 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-12 15:10:01,005 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-12 15:10:01,008 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-12 15:10:01,009 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:01,009 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-12 15:10:01,009 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-12 15:10:01,011 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-12 15:10:01,014 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/info 2023-07-12 15:10:01,014 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/info 2023-07-12 15:10:01,015 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-12 15:10:01,016 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:01,016 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-12 15:10:01,017 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:10:01,017 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/rep_barrier 2023-07-12 15:10:01,017 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-12 15:10:01,018 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:01,018 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-12 15:10:01,020 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/table 2023-07-12 15:10:01,020 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/table 2023-07-12 15:10:01,020 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-12 15:10:01,021 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:01,023 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740 2023-07-12 15:10:01,026 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740 2023-07-12 15:10:01,030 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-12 15:10:01,032 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-12 15:10:01,034 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66315578, jitterRate=-0.011820882558822632}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-12 15:10:01,034 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-12 15:10:01,045 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689174600916 2023-07-12 15:10:01,062 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-12 15:10:01,063 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-12 15:10:01,063 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,33703,1689174599345, state=OPEN 2023-07-12 15:10:01,065 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-12 15:10:01,065 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-12 15:10:01,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-12 15:10:01,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,33703,1689174599345 in 338 msec 2023-07-12 15:10:01,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-12 15:10:01,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 542 msec 2023-07-12 15:10:01,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 804 msec 2023-07-12 15:10:01,081 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689174601081, completionTime=-1 2023-07-12 15:10:01,081 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-07-12 15:10:01,081 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:01,136 DEBUG [hconnection-0x2ee4aff5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:10:01,138 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46814, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:10:01,152 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:01,152 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174661152 2023-07-12 15:10:01,152 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174721152 2023-07-12 15:10:01,152 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 71 msec 2023-07-12 15:10:01,177 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37639,1689174598377-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:01,178 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37639,1689174598377-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:01,178 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37639,1689174598377-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:01,179 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:37639, period=300000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:01,179 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-12 15:10:01,189 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-12 15:10:01,197 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-12 15:10:01,199 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-12 15:10:01,210 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:01,212 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:01,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:01,239 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,241 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7 empty. 2023-07-12 15:10:01,242 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,242 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-12 15:10:01,279 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:01,282 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 688006d0ea2ee3fc16a6d36bae9436b7, NAME => 'hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:01,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:01,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 688006d0ea2ee3fc16a6d36bae9436b7, disabling compactions & flushes 2023-07-12 15:10:01,304 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. after waiting 0 ms 2023-07-12 15:10:01,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,304 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,304 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 688006d0ea2ee3fc16a6d36bae9436b7: 2023-07-12 15:10:01,312 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:01,329 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689174601315"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174601315"}]},"ts":"1689174601315"} 2023-07-12 15:10:01,353 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:01,355 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:01,359 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174601355"}]},"ts":"1689174601355"} 2023-07-12 15:10:01,364 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-12 15:10:01,367 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:01,369 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:01,369 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:01,369 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:01,369 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:01,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=688006d0ea2ee3fc16a6d36bae9436b7, ASSIGN}] 2023-07-12 15:10:01,374 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=688006d0ea2ee3fc16a6d36bae9436b7, ASSIGN 2023-07-12 15:10:01,375 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=688006d0ea2ee3fc16a6d36bae9436b7, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:01,529 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:01,531 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:01,531 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689174601530"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174601530"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174601530"}]},"ts":"1689174601530"} 2023-07-12 15:10:01,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 688006d0ea2ee3fc16a6d36bae9436b7, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:01,694 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:01,695 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:10:01,702 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:40668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:10:01,710 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,712 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 688006d0ea2ee3fc16a6d36bae9436b7, NAME => 'hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:01,713 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:01,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:01,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,717 INFO [StoreOpener-688006d0ea2ee3fc16a6d36bae9436b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,720 DEBUG [StoreOpener-688006d0ea2ee3fc16a6d36bae9436b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/info 2023-07-12 15:10:01,720 DEBUG [StoreOpener-688006d0ea2ee3fc16a6d36bae9436b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/info 2023-07-12 15:10:01,720 INFO [StoreOpener-688006d0ea2ee3fc16a6d36bae9436b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 688006d0ea2ee3fc16a6d36bae9436b7 columnFamilyName info 2023-07-12 15:10:01,721 INFO [StoreOpener-688006d0ea2ee3fc16a6d36bae9436b7-1] regionserver.HStore(310): Store=688006d0ea2ee3fc16a6d36bae9436b7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:01,724 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,726 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,732 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:10:01,735 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:01,736 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 688006d0ea2ee3fc16a6d36bae9436b7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64521944, jitterRate=-0.03854811191558838}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:10:01,736 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 688006d0ea2ee3fc16a6d36bae9436b7: 2023-07-12 15:10:01,738 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7., pid=6, masterSystemTime=1689174601694 2023-07-12 15:10:01,742 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,743 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:10:01,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:01,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689174601744"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174601744"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174601744"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174601744"}]},"ts":"1689174601744"} 2023-07-12 15:10:01,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-12 15:10:01,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 688006d0ea2ee3fc16a6d36bae9436b7, server=jenkins-hbase20.apache.org,37993,1689174599377 in 210 msec 2023-07-12 15:10:01,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-12 15:10:01,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=688006d0ea2ee3fc16a6d36bae9436b7, ASSIGN in 381 msec 2023-07-12 15:10:01,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:01,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174601757"}]},"ts":"1689174601757"} 2023-07-12 15:10:01,759 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-12 15:10:01,761 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:01,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 562 msec 2023-07-12 15:10:01,814 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-12 15:10:01,818 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-12 15:10:01,818 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:01,843 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:10:01,846 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:40682, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:10:01,863 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:01,880 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-12 15:10:01,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 33 msec 2023-07-12 15:10:01,897 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:01,908 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-12 15:10:01,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-07-12 15:10:01,925 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-12 15:10:01,927 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-12 15:10:01,929 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.498sec 2023-07-12 15:10:01,931 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-12 15:10:01,932 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-12 15:10:01,932 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-12 15:10:01,933 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37639,1689174598377-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-12 15:10:01,934 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37639,1689174598377-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-12 15:10:01,942 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-12 15:10:02,009 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ReadOnlyZKClient(139): Connect 0x722c7441 to 127.0.0.1:63723 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-12 15:10:02,015 DEBUG [Listener at localhost.localdomain/46471] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6347cc5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-12 15:10:02,031 DEBUG [hconnection-0x32c12c41-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:10:02,042 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46818, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:10:02,051 INFO [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:10:02,080 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=363, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=171, AvailableMemoryMB=5692 2023-07-12 15:10:02,084 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:02,092 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-12 15:10:02,095 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-12 15:10:02,111 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:10:02,111 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:10:02,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:10:02,118 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,120 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:02,122 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:02,124 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,125 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 empty. 2023-07-12 15:10:02,126 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,126 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-12 15:10:02,146 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:02,148 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 729ceb1cf3f9ae336ecb337304ed6137, NAME => 'testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:02,169 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:02,170 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 729ceb1cf3f9ae336ecb337304ed6137, disabling compactions & flushes 2023-07-12 15:10:02,170 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,170 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,170 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. after waiting 0 ms 2023-07-12 15:10:02,170 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,170 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,170 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:02,174 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:02,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689174602176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174602176"}]},"ts":"1689174602176"} 2023-07-12 15:10:02,179 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:02,181 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:02,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174602181"}]},"ts":"1689174602181"} 2023-07-12 15:10:02,183 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-07-12 15:10:02,186 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:02,188 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:02,188 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:02,188 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:02,188 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:02,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, ASSIGN}] 2023-07-12 15:10:02,191 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, ASSIGN 2023-07-12 15:10:02,192 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:10:02,342 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:02,344 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=729ceb1cf3f9ae336ecb337304ed6137, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,345 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689174602344"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174602344"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174602344"}]},"ts":"1689174602344"} 2023-07-12 15:10:02,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 729ceb1cf3f9ae336ecb337304ed6137, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:02,506 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,507 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:10:02,512 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:35352, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:10:02,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 729ceb1cf3f9ae336ecb337304ed6137, NAME => 'testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:02,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:02,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:02,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,527 INFO [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,529 DEBUG [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1 2023-07-12 15:10:02,529 DEBUG [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1 2023-07-12 15:10:02,529 INFO [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 729ceb1cf3f9ae336ecb337304ed6137 columnFamilyName cf1 2023-07-12 15:10:02,530 INFO [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] regionserver.HStore(310): Store=729ceb1cf3f9ae336ecb337304ed6137/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:02,530 INFO [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,533 DEBUG [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2 2023-07-12 15:10:02,533 DEBUG [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2 2023-07-12 15:10:02,533 INFO [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 729ceb1cf3f9ae336ecb337304ed6137 columnFamilyName cf2 2023-07-12 15:10:02,534 INFO [StoreOpener-729ceb1cf3f9ae336ecb337304ed6137-1] regionserver.HStore(310): Store=729ceb1cf3f9ae336ecb337304ed6137/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:02,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:02,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:02,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:02,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 729ceb1cf3f9ae336ecb337304ed6137; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71919088, jitterRate=0.07167792320251465}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:02,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:02,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137., pid=11, masterSystemTime=1689174602506 2023-07-12 15:10:02,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,556 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=729ceb1cf3f9ae336ecb337304ed6137, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,557 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689174602556"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174602556"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174602556"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174602556"}]},"ts":"1689174602556"} 2023-07-12 15:10:02,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-12 15:10:02,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 729ceb1cf3f9ae336ecb337304ed6137, server=jenkins-hbase20.apache.org,41741,1689174599302 in 209 msec 2023-07-12 15:10:02,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-12 15:10:02,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, ASSIGN in 375 msec 2023-07-12 15:10:02,568 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:02,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174602568"}]},"ts":"1689174602568"} 2023-07-12 15:10:02,571 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-07-12 15:10:02,574 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:02,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 458 msec 2023-07-12 15:10:02,625 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:10:02,630 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:35362, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:10:02,678 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:02,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,685 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:02,685 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:02,686 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 15:10:02,686 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 15:10:02,686 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,686 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,687 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:02,687 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:02,687 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,687 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:02,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:02,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:02,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:02,688 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:02,688 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:02,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,692 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,694 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:02,695 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:02,695 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:02,695 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:02,696 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:02,697 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:02,697 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,698 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. started... 2023-07-12 15:10:02,699 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:10:02,700 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,700 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,700 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,700 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:02,700 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,701 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:02,762 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/1ba1592282fa4fccbb800fbcff4c4d84 2023-07-12 15:10:02,830 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/6d853cccbbcd490a95494dfba8a21576 2023-07-12 15:10:02,839 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/1ba1592282fa4fccbb800fbcff4c4d84 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84 2023-07-12 15:10:02,850 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:02,853 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/6d853cccbbcd490a95494dfba8a21576 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576 2023-07-12 15:10:02,861 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:02,864 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 164ms, sequenceid=6, compaction requested=false 2023-07-12 15:10:02,865 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:02,868 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:02,868 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:02,868 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:02,869 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:02,869 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,870 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:02,870 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:02,871 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,871 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:02,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:02,872 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,872 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:02,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:02,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 15:10:02,875 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:02,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b7ecc1[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:02,875 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:02,878 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:02,878 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:02,878 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:02,880 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:02,880 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,880 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:02,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:02,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:02,880 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:02,881 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:02,881 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:02,881 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:02,881 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:02,882 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:02,882 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:02,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:10:02,887 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 15:10:02,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:02,887 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:02,887 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:02,887 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,887 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:02,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:02,892 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:02,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:02,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,892 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:02,893 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:02,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,896 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,913 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:02,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:02,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:02,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:02,914 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,915 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,915 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:02,915 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,915 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:02,915 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:02,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:02,915 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:02,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:02,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:02,915 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:02,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:02,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:02,916 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:02,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:02,918 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:02,918 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:03,018 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:03,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:03,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:03,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:03,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,053 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:03,053 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,054 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 15:10:03,054 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 15:10:03,054 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,054 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,055 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,055 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,055 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,055 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,056 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,056 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,057 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,058 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,058 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,059 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,059 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,059 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,058 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,059 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,059 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,059 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,060 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,060 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:03,060 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:03,060 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:03,060 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. started... 2023-07-12 15:10:03,061 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:03,066 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,067 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,067 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,067 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:03,067 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,067 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:03,083 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 2023-07-12 15:10:03,105 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/37da2063b08b4cc4970502ca22a46762 2023-07-12 15:10:03,116 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 2023-07-12 15:10:03,124 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:03,125 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/37da2063b08b4cc4970502ca22a46762 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762 2023-07-12 15:10:03,135 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:03,136 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 75ms, sequenceid=19, compaction requested=false 2023-07-12 15:10:03,136 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:03,136 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:03,136 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:03,137 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:03,137 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,137 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,137 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,139 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,139 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:03,139 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,139 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:03,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:03,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:03,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 15:10:03,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77ee505b[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:03,142 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:03,142 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:03,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,143 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:03,143 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:03,144 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:03,143 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:03,144 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:03,144 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:03,144 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:03,144 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:03,144 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:03,146 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,146 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:03,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:03,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:03,146 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,146 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:03,146 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:03,146 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:03,147 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,147 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:03,147 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:03,148 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:03,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:03,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:03,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:10:03,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:03,154 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 15:10:03,154 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:03,154 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:03,154 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,154 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:03,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:03,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,164 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:03,164 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,165 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:03,165 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:03,167 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,168 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,168 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,176 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,175 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:03,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:03,176 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:03,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,277 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:03,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:03,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:03,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:03,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,309 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:03,309 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,310 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 15:10:03,310 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 15:10:03,310 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,310 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,311 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,311 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,311 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,311 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,312 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,312 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,313 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,313 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,314 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,313 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,314 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,314 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,314 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,315 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,315 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,314 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,315 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,316 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,316 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:03,316 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:03,316 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:03,316 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. started... 2023-07-12 15:10:03,317 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:03,317 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,318 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,318 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,318 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:03,319 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,319 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:03,350 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/0ec251b7af224e18a6364838a93a52cf 2023-07-12 15:10:03,776 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/ee4a49327a62483984b6e3882f502bf9 2023-07-12 15:10:03,792 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/0ec251b7af224e18a6364838a93a52cf as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf 2023-07-12 15:10:03,801 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:03,802 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/ee4a49327a62483984b6e3882f502bf9 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9 2023-07-12 15:10:03,811 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:03,812 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 495ms, sequenceid=32, compaction requested=true 2023-07-12 15:10:03,813 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:03,813 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:03,813 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:03,815 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:03,815 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,815 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,815 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,816 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,816 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:03,817 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,817 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:03,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:03,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 15:10:03,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45c4eb95[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:03,820 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:03,820 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,820 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,820 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,821 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:03,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:03,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:03,821 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:03,823 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:03,823 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,823 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:03,824 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:03,824 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,824 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:03,825 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,825 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:03,825 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,825 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:03,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:03,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:10:03,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:03,830 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 15:10:03,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:03,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:03,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,830 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:03,838 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,838 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:03,838 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,838 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,839 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,838 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:03,839 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,839 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,840 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:03,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,841 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:03,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,844 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,845 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,845 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:03,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,852 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:03,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,853 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:03,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,853 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:03,853 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,854 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,854 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:03,854 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:03,854 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,854 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:03,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:03,954 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:03,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:03,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:03,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:03,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,988 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:03,988 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,989 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 15:10:03,989 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 15:10:03,989 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,989 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,990 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,990 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,990 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:03,990 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,991 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:03,991 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,992 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,992 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,992 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,992 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:03,995 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,995 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,995 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,995 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:03,996 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:03,995 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:03,996 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:03,996 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:03,996 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:03,997 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:03,997 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:03,997 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:03,997 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. started... 2023-07-12 15:10:03,997 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,000 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,018 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/6e25aadab8614d4ea311c96b64c02dc8 2023-07-12 15:10:04,043 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/4ba43c7890c64d2da7d393b666b6f2d6 2023-07-12 15:10:04,052 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/6e25aadab8614d4ea311c96b64c02dc8 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8 2023-07-12 15:10:04,059 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:04,060 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/4ba43c7890c64d2da7d393b666b6f2d6 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6 2023-07-12 15:10:04,068 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:04,069 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 72ms, sequenceid=45, compaction requested=true 2023-07-12 15:10:04,070 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:04,070 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,070 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:04,071 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:04,071 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,071 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,071 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,073 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,073 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,073 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,073 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 15:10:04,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@275fdb7a[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:04,075 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:04,076 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,076 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,076 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,076 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,077 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,077 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:04,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:04,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:04,078 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,078 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,078 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,078 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,079 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,079 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,079 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,079 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,080 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:10:04,084 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 15:10:04,084 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:04,084 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:04,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:04,084 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,084 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,085 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,085 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,086 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,086 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,088 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,088 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,088 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:04,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:04,091 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:04,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,095 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:04,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,096 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,096 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,096 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,196 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:04,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:04,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:04,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:04,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,229 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:04,229 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,230 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 15:10:04,230 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 15:10:04,231 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,231 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,232 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,232 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,232 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,232 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,232 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,232 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:04,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,234 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:04,234 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,234 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,234 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,236 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:04,236 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:04,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,236 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:04,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:04,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,237 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:04,237 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:04,237 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,237 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,238 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:04,237 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,239 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:04,239 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,240 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:04,240 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:04,240 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,240 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,240 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,240 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. started... 2023-07-12 15:10:04,240 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:04,247 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,262 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/e8160c14efb4467897062c5c3ef0f487 2023-07-12 15:10:04,286 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/7b9a6be2bfd842f8b72005c1caa0de26 2023-07-12 15:10:04,294 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/e8160c14efb4467897062c5c3ef0f487 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487 2023-07-12 15:10:04,303 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:04,305 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/7b9a6be2bfd842f8b72005c1caa0de26 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26 2023-07-12 15:10:04,314 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:04,316 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 75ms, sequenceid=58, compaction requested=true 2023-07-12 15:10:04,316 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:04,316 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,316 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:04,317 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:04,317 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,317 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,317 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,318 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,318 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,318 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,318 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 15:10:04,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ab3eab4[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:04,321 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:04,321 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,321 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,322 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,322 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:04,322 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:04,323 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,323 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,323 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,324 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,324 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,324 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,324 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,324 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,325 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,325 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:10:04,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:04,332 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 15:10:04,332 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:04,332 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:04,332 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,332 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:04,333 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,333 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,333 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,333 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,333 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,334 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,335 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,336 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,338 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,338 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,339 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,348 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:04,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,348 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,349 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:04,349 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,349 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:04,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,349 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,350 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,350 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,350 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,350 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,350 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,350 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,450 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:04,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:04,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:04,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:04,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,484 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:04,484 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,484 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-12 15:10:04,484 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-12 15:10:04,485 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,485 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,486 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,486 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,486 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,486 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,486 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,487 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,487 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:04,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:04,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,488 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:04,488 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:04,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-12 15:10:04,488 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,489 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:04,489 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-12 15:10:04,488 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,489 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:04,489 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:04,490 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,489 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:04,490 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. started... 2023-07-12 15:10:04,490 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:04,490 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-12 15:10:04,490 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,490 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:04,490 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,490 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,490 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,507 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,507 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,509 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,509 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,509 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,509 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,515 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/3c986b9085cb49d49568c962ed9d0c87 2023-07-12 15:10:04,538 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 2023-07-12 15:10:04,547 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/3c986b9085cb49d49568c962ed9d0c87 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87 2023-07-12 15:10:04,555 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:04,556 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 2023-07-12 15:10:04,564 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:04,565 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 75ms, sequenceid=71, compaction requested=true 2023-07-12 15:10:04,565 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:04,565 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,565 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:04,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:04,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-12 15:10:04,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-12 15:10:04,567 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,567 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,568 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,568 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:04,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-12 15:10:04,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fda0d31[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:04,570 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:04,570 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,570 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,570 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,570 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,571 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,571 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:04,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:04,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-12 15:10:04,571 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:04,572 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,572 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,572 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,573 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,572 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,573 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,573 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-12 15:10:04,573 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,573 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,573 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-12 15:10:04,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:10:04,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:04,577 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-12 15:10:04,577 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:04,577 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:04,577 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,577 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,584 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,583 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,584 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,585 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:04,587 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,588 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,588 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,591 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:04,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-12 15:10:04,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,592 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:04,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:04,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:04,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,594 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:04,595 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-12 15:10:04,694 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:04,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:04,726 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,733 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:04,734 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:04,734 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:04,734 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:04,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, UNASSIGN}] 2023-07-12 15:10:04,743 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, UNASSIGN 2023-07-12 15:10:04,744 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=729ceb1cf3f9ae336ecb337304ed6137, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:04,744 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689174604744"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174604744"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174604744"}]},"ts":"1689174604744"} 2023-07-12 15:10:04,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 729ceb1cf3f9ae336ecb337304ed6137, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:04,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:04,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 729ceb1cf3f9ae336ecb337304ed6137, disabling compactions & flushes 2023-07-12 15:10:04,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. after waiting 0 ms 2023-07-12 15:10:04,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:04,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 729ceb1cf3f9ae336ecb337304ed6137 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:10:04,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/d3202a5c3d2744f7b8b76516e3db00f6 2023-07-12 15:10:04,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/099024bf1e8c4ecfbe2d301982222d84 2023-07-12 15:10:04,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf1/d3202a5c3d2744f7b8b76516e3db00f6 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6 2023-07-12 15:10:04,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:10:04,975 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.tmp/cf2/099024bf1e8c4ecfbe2d301982222d84 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84 2023-07-12 15:10:04,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:10:04,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 729ceb1cf3f9ae336ecb337304ed6137 in 67ms, sequenceid=82, compaction requested=true 2023-07-12 15:10:05,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:10:05,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:05,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. 2023-07-12 15:10:05,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 729ceb1cf3f9ae336ecb337304ed6137: 2023-07-12 15:10:05,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=729ceb1cf3f9ae336ecb337304ed6137, regionState=CLOSED 2023-07-12 15:10:05,014 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689174605013"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174605013"}]},"ts":"1689174605013"} 2023-07-12 15:10:05,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-12 15:10:05,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 729ceb1cf3f9ae336ecb337304ed6137, server=jenkins-hbase20.apache.org,41741,1689174599302 in 269 msec 2023-07-12 15:10:05,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-12 15:10:05,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=729ceb1cf3f9ae336ecb337304ed6137, UNASSIGN in 278 msec 2023-07-12 15:10:05,041 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=729ceb1cf3f9ae336ecb337304ed6137, threads=14 2023-07-12 15:10:05,041 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,041 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,042 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,042 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,042 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,042 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,042 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,042 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,045 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,045 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,045 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,045 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,046 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,046 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,071 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6, top=true 2023-07-12 15:10:05,074 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26, top=true 2023-07-12 15:10:05,076 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487, top=true 2023-07-12 15:10:05,080 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627, top=true 2023-07-12 15:10:05,080 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87, top=true 2023-07-12 15:10:05,123 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.splits/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6 for child: e3d4e0ad69916b0568e043baa8382198, parent: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,123 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,146 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.splits/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487 for child: e3d4e0ad69916b0568e043baa8382198, parent: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,147 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,152 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.splits/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26 for child: e3d4e0ad69916b0568e043baa8382198, parent: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,152 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,153 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84, top=true 2023-07-12 15:10:05,180 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.splits/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87 for child: e3d4e0ad69916b0568e043baa8382198, parent: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,180 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,183 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.splits/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627 for child: e3d4e0ad69916b0568e043baa8382198, parent: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,183 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,234 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/.splits/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84 for child: e3d4e0ad69916b0568e043baa8382198, parent: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,235 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,234 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,234 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,259 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,263 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,265 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,266 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,266 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,667 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84 for region: 729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:05,667 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 729ceb1cf3f9ae336ecb337304ed6137 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:10:05,718 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:10:05,720 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:10:05,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1689174605723"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1689174605723"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1689174605723"}]},"ts":"1689174605723"} 2023-07-12 15:10:05,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174605723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174605723"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174605723"}]},"ts":"1689174605723"} 2023-07-12 15:10:05,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174605723"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174605723"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174605723"}]},"ts":"1689174605723"} 2023-07-12 15:10:05,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, ASSIGN}] 2023-07-12 15:10:05,766 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, ASSIGN 2023-07-12 15:10:05,767 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:10:05,768 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, ASSIGN 2023-07-12 15:10:05,769 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:10:05,917 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:10:05,920 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=70a49d37d17b69af191be811441bc737, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:05,920 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174605920"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174605920"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174605920"}]},"ts":"1689174605920"} 2023-07-12 15:10:05,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 70a49d37d17b69af191be811441bc737, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:05,930 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=e3d4e0ad69916b0568e043baa8382198, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:05,931 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174605930"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174605930"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174605930"}]},"ts":"1689174605930"} 2023-07-12 15:10:05,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure e3d4e0ad69916b0568e043baa8382198, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:06,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e3d4e0ad69916b0568e043baa8382198, NAME => 'testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.', STARTKEY => '26', ENDKEY => ''} 2023-07-12 15:10:06,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:06,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:06,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,087 INFO [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,088 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1 2023-07-12 15:10:06,088 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1 2023-07-12 15:10:06,088 INFO [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e3d4e0ad69916b0568e043baa8382198 columnFamilyName cf1 2023-07-12 15:10:06,124 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf-top 2023-07-12 15:10:06,133 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84-top 2023-07-12 15:10:06,141 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5-top 2023-07-12 15:10:06,150 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8-top 2023-07-12 15:10:06,158 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87 2023-07-12 15:10:06,164 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6 2023-07-12 15:10:06,169 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487 2023-07-12 15:10:06,170 INFO [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(310): Store=e3d4e0ad69916b0568e043baa8382198/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:06,170 INFO [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,172 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2 2023-07-12 15:10:06,172 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2 2023-07-12 15:10:06,172 INFO [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e3d4e0ad69916b0568e043baa8382198 columnFamilyName cf2 2023-07-12 15:10:06,189 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762-top 2023-07-12 15:10:06,195 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6-top 2023-07-12 15:10:06,201 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576-top 2023-07-12 15:10:06,208 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9-top 2023-07-12 15:10:06,214 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84 2023-07-12 15:10:06,221 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627 2023-07-12 15:10:06,226 DEBUG [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26 2023-07-12 15:10:06,227 INFO [StoreOpener-e3d4e0ad69916b0568e043baa8382198-1] regionserver.HStore(310): Store=e3d4e0ad69916b0568e043baa8382198/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:06,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:06,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened e3d4e0ad69916b0568e043baa8382198; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63945940, jitterRate=-0.047131240367889404}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:06,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for e3d4e0ad69916b0568e043baa8382198: 2023-07-12 15:10:06,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198., pid=18, masterSystemTime=1689174606079 2023-07-12 15:10:06,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:06,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:06,241 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:06,241 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:06,243 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,243 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,243 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1912): e3d4e0ad69916b0568e043baa8382198/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:06,243 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1912): e3d4e0ad69916b0568e043baa8382198/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:06,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,243 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e3d4e0ad69916b0568e043baa8382198/cf2 in testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,243 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e3d4e0ad69916b0568e043baa8382198/cf1 in testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 70a49d37d17b69af191be811441bc737, NAME => 'testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.', STARTKEY => '', ENDKEY => '26'} 2023-07-12 15:10:06,244 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/.tmp, totalSize=34.9 K 2023-07-12 15:10:06,244 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/.tmp, totalSize=34.9 K 2023-07-12 15:10:06,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:06,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,245 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=e3d4e0ad69916b0568e043baa8382198, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:06,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:06,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,247 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174606245"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174606245"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174606245"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174606245"}]},"ts":"1689174606245"} 2023-07-12 15:10:06,248 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174602631 2023-07-12 15:10:06,248 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174602631 2023-07-12 15:10:06,250 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174603025 2023-07-12 15:10:06,250 INFO [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,251 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174603025 2023-07-12 15:10:06,252 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174603281 2023-07-12 15:10:06,252 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174603281 2023-07-12 15:10:06,253 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1 2023-07-12 15:10:06,253 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1 2023-07-12 15:10:06,253 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174603960 2023-07-12 15:10:06,253 INFO [StoreOpener-70a49d37d17b69af191be811441bc737-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70a49d37d17b69af191be811441bc737 columnFamilyName cf1 2023-07-12 15:10:06,254 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174604202 2023-07-12 15:10:06,254 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174603960 2023-07-12 15:10:06,255 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174604202 2023-07-12 15:10:06,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-07-12 15:10:06,256 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174604455 2023-07-12 15:10:06,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure e3d4e0ad69916b0568e043baa8382198, server=jenkins-hbase20.apache.org,41741,1689174599302 in 318 msec 2023-07-12 15:10:06,257 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174604455 2023-07-12 15:10:06,257 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174604700 2023-07-12 15:10:06,258 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174604700 2023-07-12 15:10:06,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, ASSIGN in 492 msec 2023-07-12 15:10:06,287 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] throttle.PressureAwareThroughputController(145): e3d4e0ad69916b0568e043baa8382198#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:06,289 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] throttle.PressureAwareThroughputController(145): e3d4e0ad69916b0568e043baa8382198#cf2#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:06,291 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf-bottom 2023-07-12 15:10:06,297 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84-bottom 2023-07-12 15:10:06,301 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5-bottom 2023-07-12 15:10:06,307 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8-bottom 2023-07-12 15:10:06,311 INFO [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(310): Store=70a49d37d17b69af191be811441bc737/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:06,311 INFO [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,312 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2 2023-07-12 15:10:06,312 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2 2023-07-12 15:10:06,313 INFO [StoreOpener-70a49d37d17b69af191be811441bc737-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70a49d37d17b69af191be811441bc737 columnFamilyName cf2 2023-07-12 15:10:06,316 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/.tmp/cf1/692b7b5174a146bf97044880afb37f7e as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/692b7b5174a146bf97044880afb37f7e 2023-07-12 15:10:06,324 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/.tmp/cf2/7c6666ac45664ddc9823cdf532695d27 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/7c6666ac45664ddc9823cdf532695d27 2023-07-12 15:10:06,336 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e3d4e0ad69916b0568e043baa8382198/cf1 of e3d4e0ad69916b0568e043baa8382198 into 692b7b5174a146bf97044880afb37f7e(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:06,337 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e3d4e0ad69916b0568e043baa8382198: 2023-07-12 15:10:06,337 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198., storeName=e3d4e0ad69916b0568e043baa8382198/cf1, priority=9, startTime=1689174606239; duration=0sec 2023-07-12 15:10:06,338 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762-bottom 2023-07-12 15:10:06,340 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:06,341 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e3d4e0ad69916b0568e043baa8382198/cf2 of e3d4e0ad69916b0568e043baa8382198 into 7c6666ac45664ddc9823cdf532695d27(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:06,341 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e3d4e0ad69916b0568e043baa8382198: 2023-07-12 15:10:06,341 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198., storeName=e3d4e0ad69916b0568e043baa8382198/cf2, priority=9, startTime=1689174606241; duration=0sec 2023-07-12 15:10:06,341 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:06,345 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6-bottom 2023-07-12 15:10:06,350 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576-bottom 2023-07-12 15:10:06,356 DEBUG [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9-bottom 2023-07-12 15:10:06,356 INFO [StoreOpener-70a49d37d17b69af191be811441bc737-1] regionserver.HStore(310): Store=70a49d37d17b69af191be811441bc737/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:06,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:06,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 70a49d37d17b69af191be811441bc737; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70252717, jitterRate=0.046847060322761536}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:06,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 70a49d37d17b69af191be811441bc737: 2023-07-12 15:10:06,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737., pid=17, masterSystemTime=1689174606079 2023-07-12 15:10:06,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:06,364 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-12 15:10:06,364 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-12 15:10:06,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:06,364 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,364 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,364 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1912): 70a49d37d17b69af191be811441bc737/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:06,364 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1912): 70a49d37d17b69af191be811441bc737/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:06,364 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 70a49d37d17b69af191be811441bc737/cf1 in testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,364 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 70a49d37d17b69af191be811441bc737/cf2 in testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,365 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/.tmp, totalSize=19.9 K 2023-07-12 15:10:06,365 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/.tmp, totalSize=19.9 K 2023-07-12 15:10:06,366 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174602631 2023-07-12 15:10:06,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,367 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174602631 2023-07-12 15:10:06,367 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174603025 2023-07-12 15:10:06,368 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=70a49d37d17b69af191be811441bc737, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:06,368 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174603025 2023-07-12 15:10:06,368 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174606367"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174606367"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174606367"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174606367"}]},"ts":"1689174606367"} 2023-07-12 15:10:06,368 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174603281 2023-07-12 15:10:06,368 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174603281 2023-07-12 15:10:06,369 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174603960 2023-07-12 15:10:06,369 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174603960 2023-07-12 15:10:06,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-12 15:10:06,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 70a49d37d17b69af191be811441bc737, server=jenkins-hbase20.apache.org,41741,1689174599302 in 447 msec 2023-07-12 15:10:06,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-07-12 15:10:06,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, ASSIGN in 610 msec 2023-07-12 15:10:06,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 in 1.6520 sec 2023-07-12 15:10:06,384 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] throttle.PressureAwareThroughputController(145): 70a49d37d17b69af191be811441bc737#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:06,385 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] throttle.PressureAwareThroughputController(145): 70a49d37d17b69af191be811441bc737#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:06,401 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/.tmp/cf2/1c2db8ae97ff42d4aca999df348163a0 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/1c2db8ae97ff42d4aca999df348163a0 2023-07-12 15:10:06,402 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/.tmp/cf1/be64631dd1fa431e88ad8d6f3ba979bc as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/be64631dd1fa431e88ad8d6f3ba979bc 2023-07-12 15:10:06,408 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 70a49d37d17b69af191be811441bc737/cf2 of 70a49d37d17b69af191be811441bc737 into 1c2db8ae97ff42d4aca999df348163a0(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:06,408 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 70a49d37d17b69af191be811441bc737/cf1 of 70a49d37d17b69af191be811441bc737 into be64631dd1fa431e88ad8d6f3ba979bc(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:06,408 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 70a49d37d17b69af191be811441bc737: 2023-07-12 15:10:06,408 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 70a49d37d17b69af191be811441bc737: 2023-07-12 15:10:06,408 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737., storeName=70a49d37d17b69af191be811441bc737/cf2, priority=12, startTime=1689174606364; duration=0sec 2023-07-12 15:10:06,408 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737., storeName=70a49d37d17b69af191be811441bc737/cf1, priority=12, startTime=1689174606363; duration=0sec 2023-07-12 15:10:06,408 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:06,408 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:06,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41741] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:35362 deadline: 1689174666486, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137. is not online on jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:06,526 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:10:06,601 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-12 15:10:06,602 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-12 15:10:06,627 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-12 15:10:06,641 INFO [Listener at localhost.localdomain/46471] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 15:10:06,644 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-12 15:10:06,676 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:06,687 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-07-12 15:10:06,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionUnevenDaughter 2023-07-12 15:10:06,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:06,705 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174606704"}]},"ts":"1689174606704"} 2023-07-12 15:10:06,707 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-07-12 15:10:06,708 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-07-12 15:10:06,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 15:10:06,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, UNASSIGN}] 2023-07-12 15:10:06,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, UNASSIGN 2023-07-12 15:10:06,714 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=e3d4e0ad69916b0568e043baa8382198, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:06,715 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174606714"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174606714"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174606714"}]},"ts":"1689174606714"} 2023-07-12 15:10:06,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure e3d4e0ad69916b0568e043baa8382198, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:06,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, UNASSIGN 2023-07-12 15:10:06,722 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=70a49d37d17b69af191be811441bc737, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:06,722 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174606722"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174606722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174606722"}]},"ts":"1689174606722"} 2023-07-12 15:10:06,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 70a49d37d17b69af191be811441bc737, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:06,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 15:10:06,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing e3d4e0ad69916b0568e043baa8382198, disabling compactions & flushes 2023-07-12 15:10:06,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. after waiting 1 ms 2023-07-12 15:10:06,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,885 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6] to archive 2023-07-12 15:10:06,887 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:06,893 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,896 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,899 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,900 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,904 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-e8160c14efb4467897062c5c3ef0f487 2023-07-12 15:10:06,906 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-3c986b9085cb49d49568c962ed9d0c87 2023-07-12 15:10:06,907 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-d3202a5c3d2744f7b8b76516e3db00f6 2023-07-12 15:10:06,926 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84] to archive 2023-07-12 15:10:06,927 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:06,929 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,930 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,931 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,933 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,934 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-7b9a6be2bfd842f8b72005c1caa0de26 2023-07-12 15:10:06,936 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-71ac3ce3ab4a4d98bff00b6b2dd64627 2023-07-12 15:10:06,937 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/testSplitTableRegionUnevenDaughter=729ceb1cf3f9ae336ecb337304ed6137-099024bf1e8c4ecfbe2d301982222d84 2023-07-12 15:10:06,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:10:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:06,946 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198. 2023-07-12 15:10:06,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for e3d4e0ad69916b0568e043baa8382198: 2023-07-12 15:10:06,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:06,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 70a49d37d17b69af191be811441bc737, disabling compactions & flushes 2023-07-12 15:10:06,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. after waiting 0 ms 2023-07-12 15:10:06,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,949 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=e3d4e0ad69916b0568e043baa8382198, regionState=CLOSED 2023-07-12 15:10:06,949 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174606949"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174606949"}]},"ts":"1689174606949"} 2023-07-12 15:10:06,950 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8-bottom] to archive 2023-07-12 15:10:06,951 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:06,954 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1ba1592282fa4fccbb800fbcff4c4d84.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,956 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,958 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/0ec251b7af224e18a6364838a93a52cf.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-07-12 15:10:06,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure e3d4e0ad69916b0568e043baa8382198, server=jenkins-hbase20.apache.org,41741,1689174599302 in 238 msec 2023-07-12 15:10:06,960 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/6e25aadab8614d4ea311c96b64c02dc8.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e3d4e0ad69916b0568e043baa8382198, UNASSIGN in 248 msec 2023-07-12 15:10:06,962 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6-bottom] to archive 2023-07-12 15:10:06,965 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:06,967 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/6d853cccbbcd490a95494dfba8a21576.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,969 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/37da2063b08b4cc4970502ca22a46762.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,970 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/ee4a49327a62483984b6e3882f502bf9.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,972 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/4ba43c7890c64d2da7d393b666b6f2d6.729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:06,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:10:06,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:06,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737. 2023-07-12 15:10:06,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 70a49d37d17b69af191be811441bc737: 2023-07-12 15:10:06,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 70a49d37d17b69af191be811441bc737 2023-07-12 15:10:06,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=70a49d37d17b69af191be811441bc737, regionState=CLOSED 2023-07-12 15:10:06,987 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1689174606986"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174606986"}]},"ts":"1689174606986"} 2023-07-12 15:10:06,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-07-12 15:10:06,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 70a49d37d17b69af191be811441bc737, server=jenkins-hbase20.apache.org,41741,1689174599302 in 265 msec 2023-07-12 15:10:06,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-07-12 15:10:06,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=70a49d37d17b69af191be811441bc737, UNASSIGN in 281 msec 2023-07-12 15:10:06,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174606996"}]},"ts":"1689174606996"} 2023-07-12 15:10:06,999 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-07-12 15:10:07,000 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-07-12 15:10:07,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 308 msec 2023-07-12 15:10:07,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-12 15:10:07,018 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-07-12 15:10:07,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,032 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,034 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-12 15:10:07,042 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737 2023-07-12 15:10:07,042 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:07,042 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:07,046 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/recovered.edits] 2023-07-12 15:10:07,046 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/recovered.edits] 2023-07-12 15:10:07,046 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/recovered.edits] 2023-07-12 15:10:07,051 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/692b7b5174a146bf97044880afb37f7e to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf1/692b7b5174a146bf97044880afb37f7e 2023-07-12 15:10:07,051 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/0ec251b7af224e18a6364838a93a52cf 2023-07-12 15:10:07,052 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/be64631dd1fa431e88ad8d6f3ba979bc to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf1/be64631dd1fa431e88ad8d6f3ba979bc 2023-07-12 15:10:07,053 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1ba1592282fa4fccbb800fbcff4c4d84 2023-07-12 15:10:07,055 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/7c6666ac45664ddc9823cdf532695d27 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/cf2/7c6666ac45664ddc9823cdf532695d27 2023-07-12 15:10:07,055 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/1c2db8ae97ff42d4aca999df348163a0 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/cf2/1c2db8ae97ff42d4aca999df348163a0 2023-07-12 15:10:07,055 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/1f53f39ff2624cf3ae4b9bc2630a2ed5 2023-07-12 15:10:07,057 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/3c986b9085cb49d49568c962ed9d0c87 2023-07-12 15:10:07,060 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737/recovered.edits/90.seqid 2023-07-12 15:10:07,060 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/6e25aadab8614d4ea311c96b64c02dc8 2023-07-12 15:10:07,060 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198/recovered.edits/90.seqid 2023-07-12 15:10:07,061 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/70a49d37d17b69af191be811441bc737 2023-07-12 15:10:07,061 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:07,062 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/d3202a5c3d2744f7b8b76516e3db00f6 2023-07-12 15:10:07,064 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf1/e8160c14efb4467897062c5c3ef0f487 2023-07-12 15:10:07,067 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/099024bf1e8c4ecfbe2d301982222d84 2023-07-12 15:10:07,069 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/37da2063b08b4cc4970502ca22a46762 2023-07-12 15:10:07,071 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/4ba43c7890c64d2da7d393b666b6f2d6 2023-07-12 15:10:07,073 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/6d853cccbbcd490a95494dfba8a21576 2023-07-12 15:10:07,075 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/71ac3ce3ab4a4d98bff00b6b2dd64627 2023-07-12 15:10:07,077 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/7b9a6be2bfd842f8b72005c1caa0de26 2023-07-12 15:10:07,078 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/cf2/ee4a49327a62483984b6e3882f502bf9 2023-07-12 15:10:07,081 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/recovered.edits/85.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137/recovered.edits/85.seqid 2023-07-12 15:10:07,082 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionUnevenDaughter/729ceb1cf3f9ae336ecb337304ed6137 2023-07-12 15:10:07,082 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-12 15:10:07,085 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,104 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-07-12 15:10:07,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-12 15:10:07,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-07-12 15:10:07,146 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,146 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-07-12 15:10:07,146 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174607146"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:07,146 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174607146"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:07,146 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174607146"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:07,149 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:10:07,149 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 729ceb1cf3f9ae336ecb337304ed6137, NAME => 'testSplitTableRegionUnevenDaughter,,1689174602116.729ceb1cf3f9ae336ecb337304ed6137.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 70a49d37d17b69af191be811441bc737, NAME => 'testSplitTableRegionUnevenDaughter,,1689174604726.70a49d37d17b69af191be811441bc737.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => e3d4e0ad69916b0568e043baa8382198, NAME => 'testSplitTableRegionUnevenDaughter,26,1689174604726.e3d4e0ad69916b0568e043baa8382198.', STARTKEY => '26', ENDKEY => ''}] 2023-07-12 15:10:07,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-07-12 15:10:07,150 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174607150"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:07,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-07-12 15:10:07,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:07,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 130 msec 2023-07-12 15:10:07,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-12 15:10:07,339 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-07-12 15:10:07,360 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=398 (was 363) Potentially hanging thread: RS:0;jenkins-hbase20:41741-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:43860 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ee4aff5-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:43184 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:43416 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:42896 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ee4aff5-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ee4aff5-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44194 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:33630 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44258 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44224 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ee4aff5-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:33580 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44178 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:33910 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44218 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44174 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:44180 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:43404 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1906343383_17 at /127.0.0.1:43854 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ee4aff5-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=322 (was 281) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=5581 (was 5692) 2023-07-12 15:10:07,378 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=398, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=322, ProcessCount=171, AvailableMemoryMB=5580 2023-07-12 15:10:07,379 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:07,382 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:10:07,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:10:07,384 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:07,386 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:07,387 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:07,389 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,390 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f empty. 2023-07-12 15:10:07,391 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,391 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-12 15:10:07,406 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:07,407 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 93ab9e169b318d49d9c50d2917ea323f, NAME => 'testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:07,417 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:07,417 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 93ab9e169b318d49d9c50d2917ea323f, disabling compactions & flushes 2023-07-12 15:10:07,417 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,418 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,418 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. after waiting 0 ms 2023-07-12 15:10:07,418 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,418 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,418 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:07,421 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:07,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174607422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174607422"}]},"ts":"1689174607422"} 2023-07-12 15:10:07,424 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:07,425 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:07,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174607426"}]},"ts":"1689174607426"} 2023-07-12 15:10:07,427 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-07-12 15:10:07,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:07,430 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:07,430 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:07,430 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:07,431 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:07,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, ASSIGN}] 2023-07-12 15:10:07,433 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, ASSIGN 2023-07-12 15:10:07,434 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:07,584 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:07,586 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=93ab9e169b318d49d9c50d2917ea323f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174607585"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174607585"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174607585"}]},"ts":"1689174607585"} 2023-07-12 15:10:07,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:07,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 93ab9e169b318d49d9c50d2917ea323f, NAME => 'testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:07,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:07,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:07,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,759 INFO [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,761 DEBUG [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1 2023-07-12 15:10:07,761 DEBUG [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1 2023-07-12 15:10:07,761 INFO [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93ab9e169b318d49d9c50d2917ea323f columnFamilyName cf1 2023-07-12 15:10:07,762 INFO [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] regionserver.HStore(310): Store=93ab9e169b318d49d9c50d2917ea323f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:07,762 INFO [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,764 DEBUG [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2 2023-07-12 15:10:07,764 DEBUG [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2 2023-07-12 15:10:07,765 INFO [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93ab9e169b318d49d9c50d2917ea323f columnFamilyName cf2 2023-07-12 15:10:07,766 INFO [StoreOpener-93ab9e169b318d49d9c50d2917ea323f-1] regionserver.HStore(310): Store=93ab9e169b318d49d9c50d2917ea323f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:07,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:07,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:07,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 93ab9e169b318d49d9c50d2917ea323f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67082962, jitterRate=-3.85969877243042E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:07,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:07,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f., pid=27, masterSystemTime=1689174607744 2023-07-12 15:10:07,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,778 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=93ab9e169b318d49d9c50d2917ea323f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,778 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174607778"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174607778"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174607778"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174607778"}]},"ts":"1689174607778"} 2023-07-12 15:10:07,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-07-12 15:10:07,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 in 192 msec 2023-07-12 15:10:07,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-07-12 15:10:07,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, ASSIGN in 353 msec 2023-07-12 15:10:07,787 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:07,788 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174607787"}]},"ts":"1689174607787"} 2023-07-12 15:10:07,789 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-07-12 15:10:07,792 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:07,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 409 msec 2023-07-12 15:10:07,889 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-12 15:10:07,895 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:40686, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-12 15:10:07,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:07,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 15:10:07,909 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 15:10:07,909 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:07,910 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 15:10:07,910 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 15:10:07,911 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,911 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,912 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:07,912 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:07,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:07,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:07,912 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,912 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:07,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:07,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:07,913 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,913 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:07,915 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:07,915 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:07,916 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:07,916 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:07,915 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:07,917 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:07,916 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:07,918 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:07,918 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:07,918 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:07,919 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,919 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:07,919 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:07,919 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:07,919 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:07,919 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. started... 2023-07-12 15:10:07,919 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:07,920 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:10:07,925 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,925 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,925 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,925 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:07,925 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,925 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:07,933 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/b933ce9496184efbb0990453640ea729 2023-07-12 15:10:07,955 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/978e251c2c4c4c3c92955eb0d98d47ef 2023-07-12 15:10:07,960 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/b933ce9496184efbb0990453640ea729 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 2023-07-12 15:10:07,965 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:07,966 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/978e251c2c4c4c3c92955eb0d98d47ef as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef 2023-07-12 15:10:07,972 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:07,973 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 53ms, sequenceid=6, compaction requested=false 2023-07-12 15:10:07,973 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-07-12 15:10:07,974 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:07,974 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:07,974 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:07,974 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:07,974 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,974 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:07,974 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:07,976 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,976 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:07,976 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,976 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:07,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 15:10:07,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d20d51c[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:07,979 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 15:10:07,979 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,981 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,981 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:07,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:07,981 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:07,982 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:07,982 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:07,982 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:07,982 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:07,982 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:07,983 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:07,984 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:07,984 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:07,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:07,985 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:07,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:07,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:07,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:07,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:07,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 15:10:07,990 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 15:10:07,990 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:07,990 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:07,990 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 15:10:07,990 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:07,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:07,992 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:07,993 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:07,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:07,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:07,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:07,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,994 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:07,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:07,995 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:07,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:07,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:07,997 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:07,997 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:07,997 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:07,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 15:10:08,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,006 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:08,005 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,006 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:08,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,107 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:08,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:08,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:08,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:08,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 15:10:08,143 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 15:10:08,143 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,144 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 15:10:08,144 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 15:10:08,144 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,144 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,145 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,145 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,145 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,145 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,146 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,146 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,148 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,148 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,148 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,148 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,149 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,149 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,148 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,149 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:08,150 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:08,150 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,150 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,150 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,149 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,150 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,150 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. started... 2023-07-12 15:10:08,150 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,150 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,150 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:08,158 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,163 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,163 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,163 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,164 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,164 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,166 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/dab7790d28324707a96372106c3f7d09 2023-07-12 15:10:08,181 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/3c3ca5974b2141d59d2d5981cdaddc32 2023-07-12 15:10:08,187 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/dab7790d28324707a96372106c3f7d09 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 2023-07-12 15:10:08,193 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:08,194 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/3c3ca5974b2141d59d2d5981cdaddc32 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 2023-07-12 15:10:08,200 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:08,201 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 51ms, sequenceid=19, compaction requested=false 2023-07-12 15:10:08,201 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:08,201 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,201 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:08,201 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:08,201 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,201 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,201 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,203 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,203 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,203 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,203 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 15:10:08,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@247d55c9[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:08,205 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 15:10:08,205 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:08,206 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,206 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,206 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:08,207 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,207 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,207 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,209 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:08,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 15:10:08,213 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 15:10:08,213 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:08,213 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:08,213 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 15:10:08,213 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,214 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,215 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,217 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,217 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,219 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,219 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,219 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,221 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 15:10:08,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:08,222 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,224 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,324 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:08,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:08,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:08,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:08,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 15:10:08,357 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 15:10:08,357 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,358 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 15:10:08,358 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 15:10:08,358 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,358 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,359 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,359 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,359 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,359 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,360 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,360 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,361 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,361 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,362 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,362 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,362 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,362 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,363 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,363 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,362 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,363 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,363 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,363 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,364 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,364 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:08,364 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:08,364 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,364 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. started... 2023-07-12 15:10:08,364 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:08,366 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,367 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,367 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,367 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,367 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,367 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,377 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/de3d0a02aed04fc88af2b858f1ed22af 2023-07-12 15:10:08,397 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 2023-07-12 15:10:08,402 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/de3d0a02aed04fc88af2b858f1ed22af as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af 2023-07-12 15:10:08,408 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:08,409 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 2023-07-12 15:10:08,414 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:08,415 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 51ms, sequenceid=32, compaction requested=true 2023-07-12 15:10:08,415 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:08,415 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,415 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:08,415 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:08,415 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,416 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,416 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,417 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,417 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,417 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 15:10:08,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@211eedb8[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:08,419 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 15:10:08,419 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,420 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:08,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,420 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:08,421 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:08,422 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,422 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,422 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,423 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,423 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,423 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,423 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,423 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,424 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,424 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:08,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 15:10:08,428 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 15:10:08,428 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:08,428 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:08,428 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 15:10:08,428 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,429 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,430 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,431 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,431 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,441 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,441 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,441 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 15:10:08,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:08,441 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,441 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,442 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,442 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,443 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:08,443 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:08,442 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,442 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,443 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,444 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,502 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:10:08,503 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-12 15:10:08,503 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:08,503 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-07-12 15:10:08,543 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:08,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:08,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:08,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:08,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 15:10:08,577 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 15:10:08,577 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,578 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 15:10:08,578 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 15:10:08,579 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,579 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,580 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,581 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,581 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,580 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,581 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,581 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:08,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,582 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,582 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,583 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,583 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,583 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,584 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:08,584 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,584 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. started... 2023-07-12 15:10:08,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:08,584 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,584 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,584 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:08,586 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,586 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,586 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,591 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,592 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,592 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,592 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,592 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,593 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,603 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 2023-07-12 15:10:08,618 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/e55a4d0b284f40e68453071028e7a464 2023-07-12 15:10:08,624 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 2023-07-12 15:10:08,630 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:08,631 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/e55a4d0b284f40e68453071028e7a464 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 2023-07-12 15:10:08,637 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:08,638 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 54ms, sequenceid=45, compaction requested=true 2023-07-12 15:10:08,639 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:08,639 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,639 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:08,639 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:08,639 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,639 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,639 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,640 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,640 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,641 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,641 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 15:10:08,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e62df2f[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:08,644 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 15:10:08,644 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,644 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,645 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,645 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:08,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,645 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:08,646 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,646 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,646 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,647 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,647 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,647 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,648 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,648 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,648 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,648 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:08,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 15:10:08,653 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 15:10:08,654 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:08,654 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:08,654 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 15:10:08,654 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,659 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,660 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,660 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,662 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,663 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,663 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,667 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,666 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,667 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 15:10:08,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,667 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,667 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:08,667 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,668 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:08,768 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:08,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:08,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:08,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:08,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 15:10:08,800 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 15:10:08,800 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,800 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 15:10:08,800 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 15:10:08,801 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,801 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,802 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,802 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,802 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,802 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,802 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,802 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,803 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,803 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,803 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:08,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,803 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:08,804 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:08,804 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,804 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. started... 2023-07-12 15:10:08,804 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:08,805 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,805 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,805 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,805 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,805 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,806 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,819 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/ddffc5a5c3414094a8288247a2bf78b3 2023-07-12 15:10:08,834 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/8c92b440fc194507b0107c0816ae60d9 2023-07-12 15:10:08,840 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/ddffc5a5c3414094a8288247a2bf78b3 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 2023-07-12 15:10:08,847 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:08,848 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/8c92b440fc194507b0107c0816ae60d9 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 2023-07-12 15:10:08,854 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:08,855 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 51ms, sequenceid=58, compaction requested=true 2023-07-12 15:10:08,855 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:08,855 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:08,855 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:08,855 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:08,855 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,855 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:08,855 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:08,856 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,856 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,857 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,857 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 15:10:08,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15917fca[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:08,858 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 15:10:08,858 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,860 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,860 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,859 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:08,860 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:08,861 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,861 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,862 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:08,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 15:10:08,864 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 15:10:08,864 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:08,865 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:08,865 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 15:10:08,865 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:08,872 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,872 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,872 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,872 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,872 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,873 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,872 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,873 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:08,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,873 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,874 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:08,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,876 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,876 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:08,876 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:08,883 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:08,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,884 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 15:10:08,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:08,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:08,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,883 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:08,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,884 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,885 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:08,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:08,885 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:08,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,885 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:08,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,885 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:08,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,885 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:08,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:08,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:08,885 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:08,985 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:08,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:09,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:09,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:09,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-12 15:10:09,013 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-12 15:10:09,013 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:09,013 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-12 15:10:09,013 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-12 15:10:09,013 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,013 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,014 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:09,014 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:09,014 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:09,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:09,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:09,014 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:09,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:09,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:09,015 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,015 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:09,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:09,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:09,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:09,016 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:09,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:09,017 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:09,017 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:09,017 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. started... 2023-07-12 15:10:09,017 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:09,018 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,018 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,018 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,018 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:09,019 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,019 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:09,030 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/3805d2ad4cea479f9f6ef40ef46b722d 2023-07-12 15:10:09,048 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/03ea0a3560d44ec4ad178db24ded71ca 2023-07-12 15:10:09,054 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/3805d2ad4cea479f9f6ef40ef46b722d as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d 2023-07-12 15:10:09,060 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:09,061 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/03ea0a3560d44ec4ad178db24ded71ca as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca 2023-07-12 15:10:09,067 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:09,069 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 51ms, sequenceid=71, compaction requested=true 2023-07-12 15:10:09,069 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:09,069 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:09,069 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:09,069 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:09,069 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,069 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-12 15:10:09,069 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-12 15:10:09,070 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,070 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:09,071 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,071 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:09,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-12 15:10:09,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b0ca3ac[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:09,074 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-12 15:10:09,074 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:09,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,075 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:09,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-12 15:10:09,075 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:09,076 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:09,076 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:09,077 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-12 15:10:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:09,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-12 15:10:09,082 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-12 15:10:09,082 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:09,082 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:09,082 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-12 15:10:09,082 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:09,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:09,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:09,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,091 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:09,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:09,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:09,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:09,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:09,093 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:09,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:09,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:09,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:09,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:09,094 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:09,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:09,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:09,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,097 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,098 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:09,099 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-12 15:10:09,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:09,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:09,102 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:09,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:09,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:09,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:09,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-12 15:10:09,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:09,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:09,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:09,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:09,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-12 15:10:09,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:09,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:09,103 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-12 15:10:09,104 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-12 15:10:09,204 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:09,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:09,475 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-12 15:10:09,477 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:09,479 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:09,484 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:09,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:09,730 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,732 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:09,733 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:09,733 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:09,738 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-12 15:10:09,739 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:09,739 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:09,740 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:09,740 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:09,742 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:09,745 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-12 15:10:09,747 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:09,748 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:09,749 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:09,749 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:09,749 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:09,750 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:09,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:09,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:09,752 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:09,752 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,753 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:09,753 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,753 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,753 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,754 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:10:09,754 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:09,755 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:09,756 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174639756 2023-07-12 15:10:09,758 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,758 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:09,762 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:09,762 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:09,762 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:09,763 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174669763 2023-07-12 15:10:09,763 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174729763 2023-07-12 15:10:09,763 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-12 15:10:09,763 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:09,765 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:09,765 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:09,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:09,766 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:10,013 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,013 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:10,013 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:10,013 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:10,014 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:10,015 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:10,015 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:10,015 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:10,015 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:10,016 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:10,018 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 15:10:10,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:10,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:10,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:10,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:10,020 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:10,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,022 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:10,023 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,023 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,023 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,024 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,024 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,024 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:10,025 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:10,026 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:10,026 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174640026 2023-07-12 15:10:10,027 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,027 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:10,030 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,030 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,030 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:10,030 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174670030 2023-07-12 15:10:10,031 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174730031 2023-07-12 15:10:10,031 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:10,031 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:10,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN}] 2023-07-12 15:10:10,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:10,033 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:10,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:10,281 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,281 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:10,282 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:10,282 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:10,284 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:10,285 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:10,285 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:10,285 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:10,286 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:10,288 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-07-12 15:10:10,293 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 15:10:10,293 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:10,294 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:10,294 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:10,294 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:10,295 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,295 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:10,295 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,296 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,296 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:10,296 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,296 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,296 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,296 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,297 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,297 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:10,297 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:10,298 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:10,298 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174640298 2023-07-12 15:10:10,298 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:10,299 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,302 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,302 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,302 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:10,302 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174670302 2023-07-12 15:10:10,302 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174730302 2023-07-12 15:10:10,303 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:10:10,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN}] 2023-07-12 15:10:10,303 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:10,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,306 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:10,307 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174610306"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174610306"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174610306"}]},"ts":"1689174610306"} 2023-07-12 15:10:10,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:10,308 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:10,308 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:10,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:10,553 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,553 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:10,553 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:10,553 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:10,554 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:10,554 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:10,554 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:10,555 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:10,555 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:10,556 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:10,560 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:10,560 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 15:10:10,561 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:10,562 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:10,562 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:10,563 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:10,564 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,565 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:10,565 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,566 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,566 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:10,567 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,567 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,568 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,568 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,568 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,568 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,568 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,568 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,568 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:10:10,569 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:10,569 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:10,569 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f to restore RIT 2023-07-12 15:10:10,570 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174640570 2023-07-12 15:10:10,570 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:10,570 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,573 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,573 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,573 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:10,573 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174670573 2023-07-12 15:10:10,573 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174730573 2023-07-12 15:10:10,574 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:10,574 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:10,574 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174610574"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174610574"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174610574"}]},"ts":"1689174610574"} 2023-07-12 15:10:10,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:10,576 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:10,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:10,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 93ab9e169b318d49d9c50d2917ea323f, disabling compactions & flushes 2023-07-12 15:10:10,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:10,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:10,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. after waiting 0 ms 2023-07-12 15:10:10,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:10,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 93ab9e169b318d49d9c50d2917ea323f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:10:10,739 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/90c5ca78a343488ca7392549e9d2877e 2023-07-12 15:10:10,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/d0425f5f6efb4474af1dec56cbf9c29d 2023-07-12 15:10:10,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf1/90c5ca78a343488ca7392549e9d2877e as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e 2023-07-12 15:10:10,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:10:10,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/.tmp/cf2/d0425f5f6efb4474af1dec56cbf9c29d as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d 2023-07-12 15:10:10,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:10:10,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 93ab9e169b318d49d9c50d2917ea323f in 50ms, sequenceid=82, compaction requested=true 2023-07-12 15:10:10,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:10:10,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:10,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. 2023-07-12 15:10:10,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 93ab9e169b318d49d9c50d2917ea323f: 2023-07-12 15:10:10,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:10,802 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED 2023-07-12 15:10:10,802 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174610801"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174610801"}]},"ts":"1689174610801"} 2023-07-12 15:10:10,803 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:10,803 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:10,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:10,803 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:10,824 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,824 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:10,824 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:10,824 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:10,824 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:10,824 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:10,825 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:10,825 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:10,825 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:10,825 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:10,827 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:10,828 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 15:10:10,828 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-12 15:10:10,828 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:10,829 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:10,829 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:10,829 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:10,830 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,831 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:10,831 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,831 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:10,832 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:10,832 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,833 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,833 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:10,833 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,833 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,833 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,833 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:10,833 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,834 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,834 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:10,834 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:10,834 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:10,835 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:10,835 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f to restore RIT 2023-07-12 15:10:10,836 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174640836 2023-07-12 15:10:10,836 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:10,837 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:10,839 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,840 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:10,840 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:10,840 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174670840 2023-07-12 15:10:10,840 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174730840 2023-07-12 15:10:10,840 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:10:10,840 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:10,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174610840"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174610840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174610840"}]},"ts":"1689174610840"} 2023-07-12 15:10:10,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:10,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-07-12 15:10:10,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 in 266 msec 2023-07-12 15:10:10,846 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:10,846 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:10,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:10,846 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:11,090 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,091 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:11,091 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:11,091 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:11,092 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:11,092 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:11,092 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:11,092 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:11,092 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:11,093 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:11,096 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:11,096 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 15:10:11,096 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:11,096 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:11,097 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:11,098 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:11,098 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:11,098 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:11,099 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:11,099 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:11,100 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:11,100 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:11,101 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,101 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:11,101 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:11,102 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:11,102 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,102 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,102 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,102 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:11,102 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:11,102 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:11,103 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:11,103 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:11,103 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f to restore RIT 2023-07-12 15:10:11,104 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174641104 2023-07-12 15:10:11,104 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:11,104 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:11,107 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:11,107 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:11,108 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:11,108 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174671108 2023-07-12 15:10:11,108 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174731108 2023-07-12 15:10:11,108 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:11,108 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:11,110 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:11,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-12 15:10:11,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN in 807 msec 2023-07-12 15:10:11,116 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:11,116 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:11,116 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:11,358 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,358 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:11,359 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:11,359 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:11,360 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:11,360 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:11,360 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:11,360 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:11,361 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:11,362 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:11,363 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:11,363 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:11,363 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:11,364 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:11,364 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:11,364 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:11,365 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:11,365 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:11,366 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:11,366 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:11,366 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:11,367 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:11,368 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,368 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:11,368 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:11,368 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:11,368 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:11,368 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:11,369 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:11,369 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:11,370 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:11,371 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174641371 2023-07-12 15:10:11,372 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:11,372 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,375 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:11,375 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:11,375 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:11,375 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174671375 2023-07-12 15:10:11,375 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174731375 2023-07-12 15:10:11,375 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:11,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,381 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:11,391 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=93ab9e169b318d49d9c50d2917ea323f, threads=14 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,393 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,394 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,395 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,395 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,396 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,396 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,396 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,397 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,551 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,571 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,571 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,571 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,574 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,578 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,581 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,582 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,582 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,583 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,585 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,585 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,590 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:10:11,594 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,902 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:11,902 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 93ab9e169b318d49d9c50d2917ea323f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:10:11,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:11,959 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:11,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:11,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:12,126 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,126 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:12,126 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:12,127 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:12,127 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:12,128 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:12,128 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:12,128 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:12,128 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:12,130 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:12,133 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 15:10:12,133 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:12,133 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:12,134 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:12,135 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:12,135 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:12,135 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:12,136 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,136 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:12,136 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,137 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,137 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:12,138 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,138 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:12,138 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:12,139 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:12,139 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,139 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,139 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,139 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:12,139 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:12,139 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 15:10:12,140 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:12,141 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:12,141 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174642141 2023-07-12 15:10:12,143 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:12,143 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,145 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:12,145 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:12,145 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:12,145 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174672145 2023-07-12 15:10:12,145 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174732145 2023-07-12 15:10:12,146 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:12,156 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=93ab9e169b318d49d9c50d2917ea323f, threads=14 2023-07-12 15:10:12,157 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,158 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,159 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,159 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,159 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,159 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,160 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,160 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,271 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,271 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,283 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,294 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,295 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,300 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,313 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,315 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,326 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,326 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,344 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,344 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,344 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,347 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d for region: 93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:12,347 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 93ab9e169b318d49d9c50d2917ea323f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:10:12,384 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:12,390 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:10:12,392 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:10:12,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:12,392 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:12,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:12,396 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:12,396 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:12,397 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:12,398 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 15:10:12,398 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:12,398 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:12,399 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:12,399 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:12,399 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:12,399 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:12,399 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,400 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:12,400 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,400 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,401 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:12,401 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,401 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:12,401 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:12,402 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:12,402 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,402 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,402 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,402 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:12,402 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:12,402 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-12 15:10:12,403 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:12,403 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:12,404 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174642404 2023-07-12 15:10:12,405 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:12,405 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,408 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:12,408 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:12,408 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:12,408 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174672408 2023-07-12 15:10:12,408 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174732408 2023-07-12 15:10:12,408 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:12,412 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:12,414 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:12,414 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:12,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:12,659 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:12,659 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:12,659 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:12,659 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:12,660 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:12,660 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:12,661 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:12,661 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:12,661 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:12,663 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:12,664 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 15:10:12,664 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:12,664 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:12,664 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:12,665 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:12,665 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:12,665 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:12,666 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,666 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:12,666 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,667 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:12,667 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:12,668 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,668 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN 2023-07-12 15:10:12,668 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:12,668 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:12,668 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,669 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,669 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,669 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=93ab9e169b318d49d9c50d2917ea323f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:12,669 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 93ab9e169b318d49d9c50d2917ea323f, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:12,669 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:12,670 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:12,670 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:12,670 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174642670 2023-07-12 15:10:12,671 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:12,672 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:12,674 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:12,674 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=93ab9e169b318d49d9c50d2917ea323f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:12,674 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:12,674 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174672674 2023-07-12 15:10:12,674 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174732674 2023-07-12 15:10:12,674 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:12,676 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174612676"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1689174612676"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1689174612676"}]},"ts":"1689174612676"} 2023-07-12 15:10:12,676 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174612676"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174612676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174612676"}]},"ts":"1689174612676"} 2023-07-12 15:10:12,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174612676"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174612676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174612676"}]},"ts":"1689174612676"} 2023-07-12 15:10:12,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, ASSIGN}] 2023-07-12 15:10:12,684 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, ASSIGN 2023-07-12 15:10:12,685 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:12,686 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, ASSIGN 2023-07-12 15:10:12,687 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:12,835 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:10:12,836 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=eebb53209a67cd5fe14ed980ba3856a2, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:12,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174612836"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174612836"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174612836"}]},"ts":"1689174612836"} 2023-07-12 15:10:12,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure eebb53209a67cd5fe14ed980ba3856a2, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:12,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=52c16816c786a7989acf47df11e6e54a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:12,844 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174612844"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174612844"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174612844"}]},"ts":"1689174612844"} 2023-07-12 15:10:12,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 52c16816c786a7989acf47df11e6e54a, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:12,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:12,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eebb53209a67cd5fe14ed980ba3856a2, NAME => 'testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 15:10:13,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:13,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:13,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,003 INFO [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,004 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1 2023-07-12 15:10:13,004 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1 2023-07-12 15:10:13,005 INFO [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eebb53209a67cd5fe14ed980ba3856a2 columnFamilyName cf1 2023-07-12 15:10:13,023 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d-bottom 2023-07-12 15:10:13,028 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e-bottom 2023-07-12 15:10:13,057 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729-bottom 2023-07-12 15:10:13,063 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0-bottom 2023-07-12 15:10:13,068 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09-bottom 2023-07-12 15:10:13,073 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3-bottom 2023-07-12 15:10:13,078 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af-bottom 2023-07-12 15:10:13,079 INFO [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(310): Store=eebb53209a67cd5fe14ed980ba3856a2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:13,079 INFO [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,080 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2 2023-07-12 15:10:13,080 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2 2023-07-12 15:10:13,080 INFO [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eebb53209a67cd5fe14ed980ba3856a2 columnFamilyName cf2 2023-07-12 15:10:13,098 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca-bottom 2023-07-12 15:10:13,104 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32-bottom 2023-07-12 15:10:13,110 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9-bottom 2023-07-12 15:10:13,117 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef-bottom 2023-07-12 15:10:13,123 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d-bottom 2023-07-12 15:10:13,130 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464-bottom 2023-07-12 15:10:13,135 DEBUG [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3-bottom 2023-07-12 15:10:13,135 INFO [StoreOpener-eebb53209a67cd5fe14ed980ba3856a2-1] regionserver.HStore(310): Store=eebb53209a67cd5fe14ed980ba3856a2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:13,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:13,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened eebb53209a67cd5fe14ed980ba3856a2; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59128641, jitterRate=-0.11891458928585052}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:13,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for eebb53209a67cd5fe14ed980ba3856a2: 2023-07-12 15:10:13,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2., pid=33, masterSystemTime=1689174612992 2023-07-12 15:10:13,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:13,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:13,142 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:13,142 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:13,142 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,142 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): eebb53209a67cd5fe14ed980ba3856a2/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:13,142 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,142 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): eebb53209a67cd5fe14ed980ba3856a2/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:13,142 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of eebb53209a67cd5fe14ed980ba3856a2/cf2 in testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,142 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eebb53209a67cd5fe14ed980ba3856a2/cf1 in testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,143 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/.tmp, totalSize=34.9 K 2023-07-12 15:10:13,143 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/.tmp, totalSize=34.9 K 2023-07-12 15:10:13,143 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174607895 2023-07-12 15:10:13,143 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174607895 2023-07-12 15:10:13,144 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174608112 2023-07-12 15:10:13,144 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174608112 2023-07-12 15:10:13,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 52c16816c786a7989acf47df11e6e54a, NAME => 'testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 15:10:13,144 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=eebb53209a67cd5fe14ed980ba3856a2, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:13,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:13,145 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174613144"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174613144"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174613144"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174613144"}]},"ts":"1689174613144"} 2023-07-12 15:10:13,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:13,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,145 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174608327 2023-07-12 15:10:13,145 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174608327 2023-07-12 15:10:13,146 INFO [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,146 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174608547 2023-07-12 15:10:13,146 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174608547 2023-07-12 15:10:13,147 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174608774 2023-07-12 15:10:13,147 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174608774 2023-07-12 15:10:13,147 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1 2023-07-12 15:10:13,147 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1 2023-07-12 15:10:13,147 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174608988 2023-07-12 15:10:13,148 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174608988 2023-07-12 15:10:13,148 INFO [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52c16816c786a7989acf47df11e6e54a columnFamilyName cf1 2023-07-12 15:10:13,148 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174609207 2023-07-12 15:10:13,148 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174609207 2023-07-12 15:10:13,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-07-12 15:10:13,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure eebb53209a67cd5fe14ed980ba3856a2, server=jenkins-hbase20.apache.org,37993,1689174599377 in 308 msec 2023-07-12 15:10:13,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, ASSIGN in 470 msec 2023-07-12 15:10:13,161 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): eebb53209a67cd5fe14ed980ba3856a2#cf2#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:13,164 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): eebb53209a67cd5fe14ed980ba3856a2#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:13,165 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d-top 2023-07-12 15:10:13,171 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e-top 2023-07-12 15:10:13,181 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729-top 2023-07-12 15:10:13,188 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/.tmp/cf1/050fbc5041ae437998869336363956c4 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/050fbc5041ae437998869336363956c4 2023-07-12 15:10:13,189 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/.tmp/cf2/ce060a1d27084b24951aa1f16e4e4a66 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/ce060a1d27084b24951aa1f16e4e4a66 2023-07-12 15:10:13,189 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0-top 2023-07-12 15:10:13,194 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09-top 2023-07-12 15:10:13,195 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in eebb53209a67cd5fe14ed980ba3856a2/cf1 of eebb53209a67cd5fe14ed980ba3856a2 into 050fbc5041ae437998869336363956c4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:13,195 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eebb53209a67cd5fe14ed980ba3856a2: 2023-07-12 15:10:13,195 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2., storeName=eebb53209a67cd5fe14ed980ba3856a2/cf1, priority=9, startTime=1689174613141; duration=0sec 2023-07-12 15:10:13,195 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:13,199 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3-top 2023-07-12 15:10:13,200 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in eebb53209a67cd5fe14ed980ba3856a2/cf2 of eebb53209a67cd5fe14ed980ba3856a2 into ce060a1d27084b24951aa1f16e4e4a66(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:13,200 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for eebb53209a67cd5fe14ed980ba3856a2: 2023-07-12 15:10:13,200 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2., storeName=eebb53209a67cd5fe14ed980ba3856a2/cf2, priority=9, startTime=1689174613141; duration=0sec 2023-07-12 15:10:13,200 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:13,203 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af-top 2023-07-12 15:10:13,204 INFO [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(310): Store=52c16816c786a7989acf47df11e6e54a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:13,204 INFO [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,205 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2 2023-07-12 15:10:13,205 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2 2023-07-12 15:10:13,205 INFO [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52c16816c786a7989acf47df11e6e54a columnFamilyName cf2 2023-07-12 15:10:13,223 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca-top 2023-07-12 15:10:13,227 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32-top 2023-07-12 15:10:13,231 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9-top 2023-07-12 15:10:13,235 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef-top 2023-07-12 15:10:13,240 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d-top 2023-07-12 15:10:13,244 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464-top 2023-07-12 15:10:13,248 DEBUG [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3-top 2023-07-12 15:10:13,249 INFO [StoreOpener-52c16816c786a7989acf47df11e6e54a-1] regionserver.HStore(310): Store=52c16816c786a7989acf47df11e6e54a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:13,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:13,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 52c16816c786a7989acf47df11e6e54a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73836235, jitterRate=0.1002456396818161}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:13,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 52c16816c786a7989acf47df11e6e54a: 2023-07-12 15:10:13,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a., pid=34, masterSystemTime=1689174612992 2023-07-12 15:10:13,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:13,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:13,256 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:13,256 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:13,257 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,257 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,257 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): 52c16816c786a7989acf47df11e6e54a/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:13,257 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): 52c16816c786a7989acf47df11e6e54a/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:13,257 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 52c16816c786a7989acf47df11e6e54a/cf1 in testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,257 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 52c16816c786a7989acf47df11e6e54a/cf2 in testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,257 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e-top] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/.tmp, totalSize=34.9 K 2023-07-12 15:10:13,257 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d-top] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/.tmp, totalSize=34.9 K 2023-07-12 15:10:13,258 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174607895 2023-07-12 15:10:13,258 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174607895 2023-07-12 15:10:13,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,259 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174608112 2023-07-12 15:10:13,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,259 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174608112 2023-07-12 15:10:13,259 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174608327 2023-07-12 15:10:13,259 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=52c16816c786a7989acf47df11e6e54a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:13,259 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174608327 2023-07-12 15:10:13,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174613259"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174613259"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174613259"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174613259"}]},"ts":"1689174613259"} 2023-07-12 15:10:13,260 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174608547 2023-07-12 15:10:13,260 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174608547 2023-07-12 15:10:13,260 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689174608774 2023-07-12 15:10:13,260 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689174608774 2023-07-12 15:10:13,261 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689174608988 2023-07-12 15:10:13,261 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689174608988 2023-07-12 15:10:13,261 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174609207 2023-07-12 15:10:13,261 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174609207 2023-07-12 15:10:13,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-12 15:10:13,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 52c16816c786a7989acf47df11e6e54a, server=jenkins-hbase20.apache.org,37993,1689174599377 in 415 msec 2023-07-12 15:10:13,276 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): 52c16816c786a7989acf47df11e6e54a#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:13,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-07-12 15:10:13,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, ASSIGN in 593 msec 2023-07-12 15:10:13,277 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 52c16816c786a7989acf47df11e6e54a#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:13,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a in 3.7990 sec 2023-07-12 15:10:13,299 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/.tmp/cf2/ad6eb84ec2a64b73be4fca2e21482d05 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/ad6eb84ec2a64b73be4fca2e21482d05 2023-07-12 15:10:13,299 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/.tmp/cf1/f28116fe43eb46c0af99ac5e13f1558d as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/f28116fe43eb46c0af99ac5e13f1558d 2023-07-12 15:10:13,305 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 52c16816c786a7989acf47df11e6e54a/cf2 of 52c16816c786a7989acf47df11e6e54a into ad6eb84ec2a64b73be4fca2e21482d05(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:13,305 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 52c16816c786a7989acf47df11e6e54a/cf1 of 52c16816c786a7989acf47df11e6e54a into f28116fe43eb46c0af99ac5e13f1558d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:13,305 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 52c16816c786a7989acf47df11e6e54a: 2023-07-12 15:10:13,306 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 52c16816c786a7989acf47df11e6e54a: 2023-07-12 15:10:13,306 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a., storeName=52c16816c786a7989acf47df11e6e54a/cf2, priority=9, startTime=1689174613256; duration=0sec 2023-07-12 15:10:13,306 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a., storeName=52c16816c786a7989acf47df11e6e54a/cf1, priority=9, startTime=1689174613256; duration=0sec 2023-07-12 15:10:13,306 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:13,306 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:13,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37993] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:40686 deadline: 1689174673426, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f. is not online on jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:13,549 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:10:13,561 INFO [Listener at localhost.localdomain/46471] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 15:10:13,563 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:10:13,583 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:13,584 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-07-12 15:10:13,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitWithoutPONR 2023-07-12 15:10:13,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-12 15:10:13,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174613589"}]},"ts":"1689174613589"} 2023-07-12 15:10:13,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-07-12 15:10:13,592 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-07-12 15:10:13,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, UNASSIGN}] 2023-07-12 15:10:13,593 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, UNASSIGN 2023-07-12 15:10:13,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=52c16816c786a7989acf47df11e6e54a, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:13,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174613594"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174613594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174613594"}]},"ts":"1689174613594"} 2023-07-12 15:10:13,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 52c16816c786a7989acf47df11e6e54a, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:13,598 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, UNASSIGN 2023-07-12 15:10:13,598 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=eebb53209a67cd5fe14ed980ba3856a2, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:13,598 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174613598"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174613598"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174613598"}]},"ts":"1689174613598"} 2023-07-12 15:10:13,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure eebb53209a67cd5fe14ed980ba3856a2, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:13,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-12 15:10:13,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 52c16816c786a7989acf47df11e6e54a, disabling compactions & flushes 2023-07-12 15:10:13,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. after waiting 0 ms 2023-07-12 15:10:13,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,765 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e-top] to archive 2023-07-12 15:10:13,767 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:13,768 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,769 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,771 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,772 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,773 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,774 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,775 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,777 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d-top] to archive 2023-07-12 15:10:13,778 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:13,780 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,781 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,782 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,783 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,784 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,785 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,786 DEBUG [StoreCloser-testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:10:13,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:13,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a. 2023-07-12 15:10:13,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 52c16816c786a7989acf47df11e6e54a: 2023-07-12 15:10:13,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing eebb53209a67cd5fe14ed980ba3856a2, disabling compactions & flushes 2023-07-12 15:10:13,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. after waiting 0 ms 2023-07-12 15:10:13,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,791 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=52c16816c786a7989acf47df11e6e54a, regionState=CLOSED 2023-07-12 15:10:13,791 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174613791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174613791"}]},"ts":"1689174613791"} 2023-07-12 15:10:13,793 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e-bottom] to archive 2023-07-12 15:10:13,793 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:13,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-07-12 15:10:13,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 52c16816c786a7989acf47df11e6e54a, server=jenkins-hbase20.apache.org,37993,1689174599377 in 198 msec 2023-07-12 15:10:13,795 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,797 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=52c16816c786a7989acf47df11e6e54a, UNASSIGN in 203 msec 2023-07-12 15:10:13,798 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,800 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,802 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,803 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,804 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,806 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d-bottom] to archive 2023-07-12 15:10:13,807 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:13,808 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,809 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,810 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,811 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,812 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,813 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,814 DEBUG [StoreCloser-testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:10:13,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:13,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2. 2023-07-12 15:10:13,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for eebb53209a67cd5fe14ed980ba3856a2: 2023-07-12 15:10:13,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,821 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=eebb53209a67cd5fe14ed980ba3856a2, regionState=CLOSED 2023-07-12 15:10:13,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174613821"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174613821"}]},"ts":"1689174613821"} 2023-07-12 15:10:13,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-07-12 15:10:13,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure eebb53209a67cd5fe14ed980ba3856a2, server=jenkins-hbase20.apache.org,37993,1689174599377 in 223 msec 2023-07-12 15:10:13,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-07-12 15:10:13,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eebb53209a67cd5fe14ed980ba3856a2, UNASSIGN in 233 msec 2023-07-12 15:10:13,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174613828"}]},"ts":"1689174613828"} 2023-07-12 15:10:13,830 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-07-12 15:10:13,831 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-07-12 15:10:13,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 247 msec 2023-07-12 15:10:13,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-12 15:10:13,891 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-07-12 15:10:13,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitWithoutPONR 2023-07-12 15:10:13,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,896 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,897 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-12 15:10:13,905 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,905 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,905 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,907 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/recovered.edits] 2023-07-12 15:10:13,907 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/recovered.edits] 2023-07-12 15:10:13,908 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/recovered.edits] 2023-07-12 15:10:13,910 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/050fbc5041ae437998869336363956c4 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf1/050fbc5041ae437998869336363956c4 2023-07-12 15:10:13,911 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/.regioninfo to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/.regioninfo 2023-07-12 15:10:13,911 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/3805d2ad4cea479f9f6ef40ef46b722d 2023-07-12 15:10:13,912 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/ce060a1d27084b24951aa1f16e4e4a66 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/cf2/ce060a1d27084b24951aa1f16e4e4a66 2023-07-12 15:10:13,912 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/90c5ca78a343488ca7392549e9d2877e 2023-07-12 15:10:13,913 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/b933ce9496184efbb0990453640ea729 2023-07-12 15:10:13,913 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,915 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/d92127e53ffc45dbb7ae14e3ea355bb0 2023-07-12 15:10:13,915 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,915 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/.regioninfo to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/.regioninfo 2023-07-12 15:10:13,916 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/dab7790d28324707a96372106c3f7d09 2023-07-12 15:10:13,916 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,917 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/ddffc5a5c3414094a8288247a2bf78b3 2023-07-12 15:10:13,918 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,918 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/3805d2ad4cea479f9f6ef40ef46b722d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,919 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf1/de3d0a02aed04fc88af2b858f1ed22af 2023-07-12 15:10:13,919 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,920 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/90c5ca78a343488ca7392549e9d2877e.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,920 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,921 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/b933ce9496184efbb0990453640ea729.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,922 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,923 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/03ea0a3560d44ec4ad178db24ded71ca 2023-07-12 15:10:13,923 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/d92127e53ffc45dbb7ae14e3ea355bb0.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,924 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/3c3ca5974b2141d59d2d5981cdaddc32 2023-07-12 15:10:13,924 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/dab7790d28324707a96372106c3f7d09.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,927 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,928 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/8c92b440fc194507b0107c0816ae60d9 2023-07-12 15:10:13,928 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/ddffc5a5c3414094a8288247a2bf78b3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,929 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,930 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/978e251c2c4c4c3c92955eb0d98d47ef 2023-07-12 15:10:13,930 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf1/de3d0a02aed04fc88af2b858f1ed22af.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,931 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,931 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/d0425f5f6efb4474af1dec56cbf9c29d 2023-07-12 15:10:13,932 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,933 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/e55a4d0b284f40e68453071028e7a464 2023-07-12 15:10:13,933 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/03ea0a3560d44ec4ad178db24ded71ca.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,933 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,934 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3 2023-07-12 15:10:13,934 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/3c3ca5974b2141d59d2d5981cdaddc32.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,935 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,936 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/8c92b440fc194507b0107c0816ae60d9.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,937 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/52c16816c786a7989acf47df11e6e54a/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,937 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/recovered.edits/85.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f/recovered.edits/85.seqid 2023-07-12 15:10:13,938 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/978e251c2c4c4c3c92955eb0d98d47ef.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,938 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,939 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/d0425f5f6efb4474af1dec56cbf9c29d.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,940 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/f28116fe43eb46c0af99ac5e13f1558d to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf1/f28116fe43eb46c0af99ac5e13f1558d 2023-07-12 15:10:13,941 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/e55a4d0b284f40e68453071028e7a464.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,942 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/eebb53209a67cd5fe14ed980ba3856a2/cf2/f15d77f9ee3f4043ae6e23cd1a55a6d3.93ab9e169b318d49d9c50d2917ea323f 2023-07-12 15:10:13,942 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/ad6eb84ec2a64b73be4fca2e21482d05 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/cf2/ad6eb84ec2a64b73be4fca2e21482d05 2023-07-12 15:10:13,945 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2/recovered.edits/90.seqid 2023-07-12 15:10:13,945 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a/recovered.edits/90.seqid 2023-07-12 15:10:13,946 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/eebb53209a67cd5fe14ed980ba3856a2 2023-07-12 15:10:13,946 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitWithoutPONR/52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:13,946 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-12 15:10:13,948 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,950 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-07-12 15:10:13,953 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-07-12 15:10:13,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-07-12 15:10:13,954 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174613954"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:13,955 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174613954"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:13,955 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174613954"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:13,957 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:10:13,957 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 93ab9e169b318d49d9c50d2917ea323f, NAME => 'testSplitWithoutPONR,,1689174607383.93ab9e169b318d49d9c50d2917ea323f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => eebb53209a67cd5fe14ed980ba3856a2, NAME => 'testSplitWithoutPONR,,1689174609477.eebb53209a67cd5fe14ed980ba3856a2.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 52c16816c786a7989acf47df11e6e54a, NAME => 'testSplitWithoutPONR,41,1689174609477.52c16816c786a7989acf47df11e6e54a.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 15:10:13,957 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-07-12 15:10:13,957 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174613957"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:13,958 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-07-12 15:10:13,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:13,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 68 msec 2023-07-12 15:10:13,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-12 15:10:13,999 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-07-12 15:10:14,013 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=407 (was 398) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:41624 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:60638 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:34006 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:42178 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:42896 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:60622 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:41600 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:33630 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:60158 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:42182 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:60208 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase20:37993-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:44178 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2ee4aff5-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:34072 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:44174 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:44180 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:60180 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=870 (was 867) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=273 (was 322), ProcessCount=167 (was 171), AvailableMemoryMB=5428 (was 5580) 2023-07-12 15:10:14,029 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=407, OpenFileDescriptor=870, MaxFileDescriptor=60000, SystemLoadAverage=273, ProcessCount=167, AvailableMemoryMB=5428 2023-07-12 15:10:14,029 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:14,033 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:10:14,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:10:14,034 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:14,036 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:14,037 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:14,038 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,039 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 empty. 2023-07-12 15:10:14,039 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,039 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-12 15:10:14,049 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:14,050 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1eef801734bac1a9fe215d8599908306, NAME => 'testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:14,059 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:14,059 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 1eef801734bac1a9fe215d8599908306, disabling compactions & flushes 2023-07-12 15:10:14,059 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,059 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,059 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. after waiting 0 ms 2023-07-12 15:10:14,059 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,059 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,059 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:14,061 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:14,062 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174614062"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174614062"}]},"ts":"1689174614062"} 2023-07-12 15:10:14,064 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:14,065 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:14,065 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174614065"}]},"ts":"1689174614065"} 2023-07-12 15:10:14,066 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-12 15:10:14,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:14,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:14,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:14,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:14,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:14,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN}] 2023-07-12 15:10:14,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:14,073 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:14,223 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:14,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174614225"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174614225"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174614225"}]},"ts":"1689174614225"} 2023-07-12 15:10:14,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:14,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1eef801734bac1a9fe215d8599908306, NAME => 'testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:14,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:14,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:14,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,393 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,395 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1 2023-07-12 15:10:14,395 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1 2023-07-12 15:10:14,396 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1eef801734bac1a9fe215d8599908306 columnFamilyName cf1 2023-07-12 15:10:14,397 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(310): Store=1eef801734bac1a9fe215d8599908306/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:14,397 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,399 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2 2023-07-12 15:10:14,399 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2 2023-07-12 15:10:14,399 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1eef801734bac1a9fe215d8599908306 columnFamilyName cf2 2023-07-12 15:10:14,400 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(310): Store=1eef801734bac1a9fe215d8599908306/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:14,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:14,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:14,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:14,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1eef801734bac1a9fe215d8599908306; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68587544, jitterRate=0.022034049034118652}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:14,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:14,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306., pid=43, masterSystemTime=1689174614383 2023-07-12 15:10:14,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,415 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,416 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174614415"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174614415"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174614415"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174614415"}]},"ts":"1689174614415"} 2023-07-12 15:10:14,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-07-12 15:10:14,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 in 187 msec 2023-07-12 15:10:14,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-07-12 15:10:14,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN in 351 msec 2023-07-12 15:10:14,426 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:14,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174614427"}]},"ts":"1689174614427"} 2023-07-12 15:10:14,431 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-12 15:10:14,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:14,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 399 msec 2023-07-12 15:10:14,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:14,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 15:10:14,554 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 15:10:14,554 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,554 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 15:10:14,554 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:10:14,554 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,554 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,555 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,555 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,555 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,555 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,556 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:14,557 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:14,557 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,557 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. started... 2023-07-12 15:10:14,558 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:10:14,558 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,558 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,558 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,559 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,559 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,559 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,567 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/380a943e54cb457f8056f001d0c0a4e6 2023-07-12 15:10:14,578 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/50f3de1f42554b429917371f759fe52a 2023-07-12 15:10:14,583 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/380a943e54cb457f8056f001d0c0a4e6 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 2023-07-12 15:10:14,588 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:14,589 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/50f3de1f42554b429917371f759fe52a as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a 2023-07-12 15:10:14,594 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:14,595 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 37ms, sequenceid=6, compaction requested=false 2023-07-12 15:10:14,595 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-07-12 15:10:14,596 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:14,596 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:14,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:14,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,597 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,598 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,598 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 15:10:14,599 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:10:14,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@da9e658[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:14,599 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,600 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,600 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:14,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,600 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:14,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:14,601 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,601 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,602 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:14,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:10:14,604 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 15:10:14,604 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:14,605 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:14,605 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 15:10:14,605 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:14,605 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,606 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,606 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,607 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,608 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,609 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,609 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 15:10:14,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:14,611 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:14,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,611 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:14,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,612 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,712 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:14,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:14,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:14,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:14,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 15:10:14,733 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 15:10:14,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 15:10:14,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:10:14,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,734 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,734 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,734 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,734 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,735 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,735 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:14,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:14,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,736 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,737 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:14,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,738 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:14,738 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. started... 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,738 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,738 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,740 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,740 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,740 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,752 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/136d2acb5f2a41569acf48b43f2115b5 2023-07-12 15:10:14,764 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/56624e3549714c2db2521a397d6f07e8 2023-07-12 15:10:14,769 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/136d2acb5f2a41569acf48b43f2115b5 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 2023-07-12 15:10:14,776 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:14,777 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/56624e3549714c2db2521a397d6f07e8 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 2023-07-12 15:10:14,783 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:14,784 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 46ms, sequenceid=19, compaction requested=false 2023-07-12 15:10:14,785 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:14,785 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,785 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:14,785 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:14,785 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,785 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,785 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,786 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,787 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,787 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,787 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 15:10:14,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75cadb65[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:14,790 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:10:14,790 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,791 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,791 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,791 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,791 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,792 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,791 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:14,792 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:14,792 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,794 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,794 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,795 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:14,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:10:14,797 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 15:10:14,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:14,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:14,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 15:10:14,797 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:14,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,799 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,799 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,800 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,801 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,801 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,801 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,803 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 15:10:14,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,803 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:14,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,804 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,904 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:14,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:14,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:14,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:14,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 15:10:14,927 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 15:10:14,927 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,927 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 15:10:14,927 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:10:14,928 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,928 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,929 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,929 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,929 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,929 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:14,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:14,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:14,930 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,930 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:14,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:14,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:14,932 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,933 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:14,934 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,934 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,934 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,934 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. started... 2023-07-12 15:10:14,934 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:14,935 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,936 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,936 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,936 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,936 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,936 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,951 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/51c39dee82154f558039ce7a1fc7ca28 2023-07-12 15:10:14,968 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/f60323368a484d5ba92c05b5811df22b 2023-07-12 15:10:14,972 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/51c39dee82154f558039ce7a1fc7ca28 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 2023-07-12 15:10:14,977 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:14,978 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/f60323368a484d5ba92c05b5811df22b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b 2023-07-12 15:10:14,983 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:14,984 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 50ms, sequenceid=32, compaction requested=true 2023-07-12 15:10:14,984 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:14,985 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:14,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:14,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:14,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:14,985 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:14,986 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,986 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,986 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,986 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 15:10:14,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bbb8539[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:14,988 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:10:14,988 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,989 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,989 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:14,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:14,989 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,989 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:14,990 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,990 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:14,990 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:14,990 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:14,990 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:14,991 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,991 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,991 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,991 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,991 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,991 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:14,992 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,992 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:14,992 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,992 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:14,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:10:14,994 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 15:10:14,994 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:14,994 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:14,995 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 15:10:14,995 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:14,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,996 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,996 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:14,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,997 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,997 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,999 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:14,999 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:14,999 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:14,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:15,006 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:15,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:15,006 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,007 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,107 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:15,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:15,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:15,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:15,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 15:10:15,131 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,131 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,131 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 15:10:15,131 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:10:15,132 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,132 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,133 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,133 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,133 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,133 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,133 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,133 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,134 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,135 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:15,135 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:15,136 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. started... 2023-07-12 15:10:15,136 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:15,136 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,136 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,137 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,137 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,137 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,137 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,146 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/db2ee8942a904e5ba4c8f832b6947f32 2023-07-12 15:10:15,160 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/82209269bda7498085a5baccf80cfb0f 2023-07-12 15:10:15,164 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/db2ee8942a904e5ba4c8f832b6947f32 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 2023-07-12 15:10:15,168 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:15,169 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/82209269bda7498085a5baccf80cfb0f as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f 2023-07-12 15:10:15,174 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:15,175 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 39ms, sequenceid=45, compaction requested=true 2023-07-12 15:10:15,175 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:15,175 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:15,175 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:15,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:15,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,177 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,177 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,177 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,177 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 15:10:15,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55144a21[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:15,180 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:10:15,180 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,181 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,182 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:15,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,182 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:15,183 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,183 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,183 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,183 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,183 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,183 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,184 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,184 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,184 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,184 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:15,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:10:15,189 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 15:10:15,189 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:15,189 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:15,189 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 15:10:15,189 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:15,190 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,190 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,190 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,190 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,190 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,191 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,191 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,191 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,192 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,192 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,195 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,196 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,196 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,199 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:15,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,200 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:15,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,200 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,199 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,200 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:15,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,200 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,200 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,201 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,201 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,201 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,300 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:15,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:15,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:15,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:15,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 15:10:15,324 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,324 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,324 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 15:10:15,324 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:10:15,324 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,325 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,325 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,325 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,325 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,325 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,326 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,326 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:15,327 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,327 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. started... 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,327 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,328 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,328 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,328 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,328 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:15,328 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,328 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,329 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,329 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,329 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,329 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,329 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,337 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/f18976c407154b2591c4749ca57f59f7 2023-07-12 15:10:15,352 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/9b526be88a1743169583c0a75e939c4b 2023-07-12 15:10:15,358 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/f18976c407154b2591c4749ca57f59f7 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 2023-07-12 15:10:15,364 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:15,365 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/9b526be88a1743169583c0a75e939c4b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b 2023-07-12 15:10:15,371 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:15,372 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 44ms, sequenceid=58, compaction requested=true 2023-07-12 15:10:15,372 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:15,372 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:15,372 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:15,372 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:15,372 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,373 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,373 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,375 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,375 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,375 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,375 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 15:10:15,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17cdfdfd[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:15,379 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:10:15,379 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,380 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,381 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,381 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:15,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,380 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:15,381 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,381 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,381 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,381 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,381 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:15,381 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:15,383 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,383 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,383 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,384 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,384 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,384 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,384 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,384 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,384 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:15,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:10:15,390 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 15:10:15,391 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:15,391 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:15,391 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 15:10:15,391 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:15,400 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,400 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,400 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,400 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,400 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,401 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,401 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,401 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,403 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,403 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,404 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,404 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,404 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,406 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,407 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,507 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:15,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:15,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:15,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:15,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-12 15:10:15,537 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,537 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,537 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-12 15:10:15,537 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:10:15,538 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,538 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,539 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,539 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,539 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,540 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,540 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-12 15:10:15,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-12 15:10:15,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:15,541 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:15,541 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. started... 2023-07-12 15:10:15,541 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:10:15,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,542 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:15,542 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,543 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,543 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,543 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,543 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,543 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,552 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/ad005e70a51146f4aa620f8297c493db 2023-07-12 15:10:15,568 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/1edcbef678234264858296a7273ae986 2023-07-12 15:10:15,575 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/ad005e70a51146f4aa620f8297c493db as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db 2023-07-12 15:10:15,580 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:15,581 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/1edcbef678234264858296a7273ae986 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 2023-07-12 15:10:15,585 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:15,586 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 44ms, sequenceid=71, compaction requested=true 2023-07-12 15:10:15,586 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:15,586 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:15,587 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:15,587 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:15,587 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,587 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-12 15:10:15,587 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-12 15:10:15,588 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,588 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,588 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,588 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:15,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-12 15:10:15,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@347f1d92[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:15,590 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:10:15,590 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,591 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,591 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:15,591 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:15,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:15,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,592 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,593 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,593 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,593 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,593 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-12 15:10:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,593 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,593 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-12 15:10:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:15,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:10:15,596 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-12 15:10:15,596 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:15,596 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:15,596 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-12 15:10:15,596 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:15,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:15,598 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,597 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,598 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,598 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:15,600 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-12 15:10:15,600 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,601 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-12 15:10:15,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,603 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:15,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:15,603 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:15,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-12 15:10:15,604 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-12 15:10:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:15,704 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:15,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:15,971 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-12 15:10:15,972 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:15,974 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:15,976 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:15,976 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:15,976 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:15,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:15,978 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:15,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:15,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:16,224 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,225 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:16,225 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:16,225 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:16,225 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-07-12 15:10:16,226 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:16,226 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:16,226 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:16,226 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:16,227 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:16,230 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-12 15:10:16,230 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:16,231 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:16,232 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:16,232 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:16,232 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,233 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:16,233 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,234 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,234 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,235 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:16,235 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,235 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,236 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:16,237 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,237 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:16,237 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,237 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,237 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,238 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 15:10:16,239 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:16,239 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:16,240 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174646240 2023-07-12 15:10:16,240 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:16,242 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,244 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:16,244 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:16,244 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:16,244 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174676244 2023-07-12 15:10:16,244 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174736244 2023-07-12 15:10:16,244 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:10:16,244 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:16,247 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:16,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:16,248 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:16,495 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,495 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:16,495 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:16,495 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:16,496 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:16,496 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:16,497 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:16,497 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:16,497 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:16,499 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:16,502 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 15:10:16,503 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:16,503 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:16,503 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:16,503 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:16,504 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,504 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:16,504 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,504 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,505 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,505 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:16,505 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,505 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,505 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:16,506 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,506 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:16,506 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,506 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,506 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,506 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:16,507 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:16,507 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:16,508 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174646508 2023-07-12 15:10:16,509 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:16,509 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,510 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:16,511 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:16,511 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:16,511 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174676511 2023-07-12 15:10:16,511 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174736511 2023-07-12 15:10:16,511 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:16,511 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:16,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN}] 2023-07-12 15:10:16,513 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:16,513 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:16,513 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:16,761 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,762 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:16,762 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:16,762 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:16,763 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:16,763 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:16,764 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:16,764 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:16,764 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:16,766 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:16,767 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 15:10:16,768 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:16,768 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:16,769 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:16,769 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:16,769 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,770 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:16,770 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,770 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:16,771 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,771 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:16,772 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,773 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:16,773 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:16,774 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,774 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:16,774 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,774 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,774 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,775 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:16,776 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:16,777 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:16,777 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174646777 2023-07-12 15:10:16,778 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:16,778 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:16,780 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:16,780 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:16,781 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:16,781 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174676781 2023-07-12 15:10:16,781 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174736781 2023-07-12 15:10:16,781 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:16,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN}] 2023-07-12 15:10:16,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:16,783 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:16,784 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:16,784 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174616783"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174616783"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174616783"}]},"ts":"1689174616783"} 2023-07-12 15:10:16,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:16,785 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:16,785 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:16,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:16,785 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:17,031 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,032 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:17,032 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:17,032 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:17,033 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:17,033 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:17,033 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:17,033 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:17,034 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:17,036 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:17,039 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:17,040 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 15:10:17,040 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:17,041 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:17,041 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:17,041 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:17,042 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,043 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:17,043 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,044 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,045 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,045 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:17,046 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,046 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,047 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:17,047 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,048 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,048 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:17,048 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,048 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,048 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,048 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:17,048 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,048 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-07-12 15:10:17,049 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:17,050 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:17,050 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:17,050 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174647050 2023-07-12 15:10:17,051 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:17,051 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,053 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,053 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,053 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:17,053 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174677053 2023-07-12 15:10:17,053 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174737053 2023-07-12 15:10:17,053 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:17,054 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,054 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174617053"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174617053"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174617053"}]},"ts":"1689174617053"} 2023-07-12 15:10:17,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:17,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:17,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1eef801734bac1a9fe215d8599908306, disabling compactions & flushes 2023-07-12 15:10:17,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:17,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:17,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. after waiting 0 ms 2023-07-12 15:10:17,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:17,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 1eef801734bac1a9fe215d8599908306 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:10:17,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/5075de4a740045acb884c03caa40f191 2023-07-12 15:10:17,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/56535eba61724ef183e575a8153a8dbd 2023-07-12 15:10:17,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/5075de4a740045acb884c03caa40f191 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 2023-07-12 15:10:17,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:10:17,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/56535eba61724ef183e575a8153a8dbd as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd 2023-07-12 15:10:17,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:10:17,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1eef801734bac1a9fe215d8599908306 in 49ms, sequenceid=82, compaction requested=true 2023-07-12 15:10:17,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:10:17,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:17,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:17,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:17,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,267 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=CLOSED 2023-07-12 15:10:17,267 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174617267"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174617267"}]},"ts":"1689174617267"} 2023-07-12 15:10:17,269 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:17,269 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:17,269 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:17,304 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,304 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:17,304 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:17,304 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:17,305 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:17,305 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:17,305 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:17,305 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:17,305 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:17,307 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:17,309 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:17,309 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 15:10:17,309 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-12 15:10:17,310 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:17,311 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:17,311 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:17,311 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:17,312 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,312 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:17,313 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,313 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,314 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,314 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:17,315 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,315 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,316 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:17,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:17,318 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,318 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,318 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,318 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:17,318 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,318 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:17,319 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:10:17,319 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:17,320 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:17,320 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:17,320 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174647320 2023-07-12 15:10:17,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,321 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:17,324 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,324 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,324 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:17,324 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174677324 2023-07-12 15:10:17,324 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174737324 2023-07-12 15:10:17,325 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:17,325 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,325 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174617325"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174617325"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174617325"}]},"ts":"1689174617325"} 2023-07-12 15:10:17,326 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:17,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-07-12 15:10:17,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 in 271 msec 2023-07-12 15:10:17,330 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:17,330 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:17,330 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:17,575 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,575 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:17,576 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:17,576 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:17,576 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:17,577 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:17,577 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:17,577 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:17,578 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:17,579 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:17,581 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:17,581 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-12 15:10:17,581 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:17,582 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:17,583 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:17,583 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:17,583 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:17,584 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,584 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:17,585 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,585 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,586 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,586 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:17,586 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,587 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,587 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:17,587 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,587 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,587 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,588 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:17,588 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,588 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,588 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,588 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:17,588 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:17,588 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:17,589 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:17,589 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:17,589 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:17,590 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174647590 2023-07-12 15:10:17,590 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,590 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:17,593 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,593 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,593 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:17,593 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174677593 2023-07-12 15:10:17,593 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174737593 2023-07-12 15:10:17,593 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:17,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,595 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:17,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-07-12 15:10:17,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN in 814 msec 2023-07-12 15:10:17,602 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:17,602 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:17,602 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:17,844 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,844 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:17,844 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:17,844 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:17,848 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:17,848 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:17,848 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:17,848 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:17,849 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:17,850 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:17,854 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:17,854 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:17,854 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:17,855 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:17,856 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:17,857 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:17,857 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:17,858 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,859 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:17,859 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,859 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:17,860 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,860 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:17,860 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,860 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:17,861 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:17,861 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 15:10:17,862 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:17,863 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:17,863 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174647863 2023-07-12 15:10:17,864 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,864 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:17,867 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,867 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:17,867 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:17,867 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174677867 2023-07-12 15:10:17,867 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174737867 2023-07-12 15:10:17,868 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:17,868 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,868 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,868 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:17,873 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:17,881 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=1eef801734bac1a9fe215d8599908306, threads=14 2023-07-12 15:10:17,882 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,882 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,882 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,882 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,883 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,884 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:17,884 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,029 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,059 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,075 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,075 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,084 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,103 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,103 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,108 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,108 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,109 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,111 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,111 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,113 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,116 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,116 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 1eef801734bac1a9fe215d8599908306 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:10:18,147 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:18,147 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:18,147 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:18,368 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,368 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:18,368 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:18,369 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:18,369 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:18,370 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:18,370 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:18,370 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:18,370 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:18,372 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:18,376 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 15:10:18,377 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:18,377 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:18,378 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:18,379 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:18,379 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:18,379 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:18,379 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:18,380 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:18,380 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:18,380 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:18,380 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:18,380 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:18,381 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:18,381 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:18,381 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:18,381 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,381 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:18,382 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:18,382 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:18,382 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,382 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,382 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,382 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:18,382 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:18,382 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 15:10:18,383 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:18,383 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:18,384 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174648384 2023-07-12 15:10:18,385 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,385 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:18,388 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:18,388 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:18,388 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:18,388 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174678388 2023-07-12 15:10:18,388 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174738388 2023-07-12 15:10:18,388 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:18,398 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=1eef801734bac1a9fe215d8599908306, threads=14 2023-07-12 15:10:18,399 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,399 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,400 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,402 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,403 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,406 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,407 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,407 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,407 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,407 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,408 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,410 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,411 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,414 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,503 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:10:18,532 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,539 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,542 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,544 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,544 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,544 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,544 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,565 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,571 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,585 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,587 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,595 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,603 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,606 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a for region: 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:18,606 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 1eef801734bac1a9fe215d8599908306 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:10:18,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:18,647 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/8ed48bfdabff1ec7a9bd902580a8ca94/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:10:18,649 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/8f66539c980ad08fef74cce889cb8e16/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:10:18,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:18,649 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:18,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:18,888 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,889 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:18,889 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:18,889 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:18,889 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:18,889 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:18,889 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:18,889 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:18,890 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:18,891 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:18,893 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 15:10:18,893 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:18,893 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:18,893 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:18,894 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:18,895 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:18,895 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:18,896 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:18,896 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:18,897 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:18,897 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:18,898 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:18,899 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:18,899 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:18,899 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:18,899 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:18,900 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,900 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:18,900 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:18,902 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:18,902 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,902 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,902 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,902 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:18,902 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:18,903 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:10:18,903 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:18,904 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:18,904 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174648904 2023-07-12 15:10:18,905 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:18,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,908 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:18,908 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:18,908 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:18,908 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174678908 2023-07-12 15:10:18,908 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174738908 2023-07-12 15:10:18,908 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:18,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:18,914 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:18,914 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:18,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:18,914 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:19,159 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,159 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:19,159 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:19,160 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:19,160 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:19,160 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:19,161 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:19,161 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:19,161 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:19,162 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:19,164 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 15:10:19,164 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:19,164 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:19,165 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:19,166 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:19,166 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:19,166 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:19,167 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,168 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:19,168 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,168 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,169 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,170 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:19,170 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,171 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,172 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:19,173 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,173 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:19,173 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:19,173 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:19,173 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,173 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,174 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,174 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,174 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,174 DEBUG [Listener at localhost.localdomain/46471] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,174 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:10:19,175 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:19,175 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:19,176 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174649176 2023-07-12 15:10:19,177 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:19,178 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,180 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,180 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,180 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:19,181 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174679181 2023-07-12 15:10:19,181 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174739181 2023-07-12 15:10:19,181 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:10:19,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:19,193 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:19,193 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:19,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:19,193 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:19,433 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,433 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:19,433 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:19,434 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:19,434 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:19,434 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:19,435 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:19,435 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:19,435 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:19,437 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:19,440 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-12 15:10:19,440 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:19,440 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:19,441 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:19,442 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:19,442 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:19,442 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:19,443 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,443 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:19,443 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,443 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,443 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,444 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:19,444 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,444 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,445 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:19,445 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,445 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:19,445 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:19,446 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:19,446 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,446 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,446 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,446 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,446 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,446 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:19,447 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:19,447 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:19,448 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174649447 2023-07-12 15:10:19,450 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:19,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,452 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,452 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,453 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:19,453 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174679453 2023-07-12 15:10:19,453 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174739453 2023-07-12 15:10:19,453 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:19,453 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:19,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:19,456 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:19,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:19,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:19,704 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,704 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:19,704 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:19,704 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:19,705 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:19,705 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:19,705 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:19,705 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:19,705 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:19,706 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:19,708 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 15:10:19,708 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:19,708 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:19,709 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:19,709 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:19,710 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:19,710 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:19,710 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,711 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:19,711 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,711 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,712 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,712 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:19,712 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,713 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,713 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:19,714 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,714 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:19,714 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:19,714 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:19,714 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,714 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,715 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,715 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,715 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,715 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-12 15:10:19,715 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:19,716 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:19,716 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174649716 2023-07-12 15:10:19,717 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:19,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,720 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,720 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,720 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:19,720 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174679720 2023-07-12 15:10:19,720 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174739720 2023-07-12 15:10:19,720 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:19,720 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:19,722 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:19,722 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:19,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:19,722 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:19,970 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,971 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:19,971 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:19,971 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:19,972 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:19,972 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:19,972 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:19,972 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:19,973 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:19,975 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:19,977 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 15:10:19,977 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:19,978 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:19,978 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:19,979 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:19,980 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:19,980 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:19,981 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,982 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:19,982 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,982 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:19,983 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,983 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:19,983 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,983 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:19,983 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:19,984 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,984 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:19,984 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:19,984 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:19,984 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,984 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,984 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,984 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,984 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:19,984 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-12 15:10:19,985 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:19,985 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:19,986 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174649986 2023-07-12 15:10:19,987 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:19,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:19,989 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,989 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:19,989 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:19,989 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174679989 2023-07-12 15:10:19,989 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174739989 2023-07-12 15:10:19,989 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:19,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:19,993 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN] 2023-07-12 15:10:19,993 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:19,993 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:19,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:19,993 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:20,240 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,240 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:20,240 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:20,240 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:20,240 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:20,240 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:20,240 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:20,241 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:20,241 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:20,241 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:20,242 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 15:10:20,242 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:10:20,243 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:10:20,243 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 15:10:20,243 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:20,243 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:20,243 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:20,244 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:20,244 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,244 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:20,244 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,244 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,245 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,245 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:20,245 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,245 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,246 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:20,247 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,247 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:20,247 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:20,247 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:20,247 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:20,247 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,248 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,248 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,248 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:20,248 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:20,248 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:20,248 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:20,249 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:20,249 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:20,249 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:20,250 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174650250 2023-07-12 15:10:20,251 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:20,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,253 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:20,255 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:20,255 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:20,255 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:20,255 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174680255 2023-07-12 15:10:20,255 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174740255 2023-07-12 15:10:20,255 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:10:20,256 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:20,261 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:20,261 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:20,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:20,261 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:20,506 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:20,506 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:20,507 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:20,508 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:10:20,508 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 15:10:20,508 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:20,509 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:20,509 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:20,509 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:20,510 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,510 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:20,510 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,510 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,511 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,511 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:20,511 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,511 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,512 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:20,512 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,512 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:20,513 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:20,513 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,513 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,513 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,513 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:20,513 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:20,514 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:20,515 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:20,515 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:20,516 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174650516 2023-07-12 15:10:20,518 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:20,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:20,521 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:20,522 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:20,522 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:20,522 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174680522 2023-07-12 15:10:20,522 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174740522 2023-07-12 15:10:20,522 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:10:20,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:20,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:20,524 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:20,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:20,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:20,772 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,773 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:20,773 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:20,773 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:20,774 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:20,774 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:20,774 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:20,774 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:20,775 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:20,777 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:10:20,779 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 15:10:20,779 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 15:10:20,780 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:20,781 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:20,781 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:20,782 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:20,783 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,784 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:20,784 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,785 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:20,786 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,786 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:20,786 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,787 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:20,787 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:20,787 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,787 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:20,787 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:20,787 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,787 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,787 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,787 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:20,788 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 15:10:20,788 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:20,789 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:20,789 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:20,789 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174650789 2023-07-12 15:10:20,790 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:20,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:20,792 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:20,793 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:20,794 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:20,794 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:20,794 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174680794 2023-07-12 15:10:20,794 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174740794 2023-07-12 15:10:20,794 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:10:20,794 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:20,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:20,796 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-12 15:10:20,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:20,797 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:21,044 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,044 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:21,044 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:21,044 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:21,045 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:21,045 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:21,045 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:21,045 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:21,045 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:21,046 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:21,046 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 15:10:21,047 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-12 15:10:21,047 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:21,047 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:21,047 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:21,047 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:21,048 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:21,048 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:21,048 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:21,049 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:21,049 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:21,049 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:21,050 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:21,050 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:21,050 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,051 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,051 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,051 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:21,051 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,052 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,052 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,052 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:10:21,052 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:21,053 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:21,053 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:21,053 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:21,054 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174651054 2023-07-12 15:10:21,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,056 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:21,058 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,061 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:21,061 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:21,061 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:21,061 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174681061 2023-07-12 15:10:21,061 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174741061 2023-07-12 15:10:21,061 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-12 15:10:21,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:10:21,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 exec-time=5.0890 sec 2023-07-12 15:10:21,064 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,065 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=true 2023-07-12 15:10:21,215 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:21,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:21,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174621218"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174621218"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174621218"}]},"ts":"1689174621218"} 2023-07-12 15:10:21,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:21,222 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:10:21,222 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:10:21,222 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:10:21,311 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:21,311 WARN [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:10:21,312 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:10:21,312 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:10:21,313 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:10:21,314 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-07-12 15:10:21,314 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:10:21,315 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:10:21,315 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:10:21,315 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:10:21,316 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:21,316 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:10:21,316 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:21,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:10:21,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:21,317 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:10:21,318 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:21,318 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:10:21,318 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,319 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:10:21,319 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,319 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:10:21,319 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,320 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:10:21,321 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:10:21,322 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:10:21,322 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306 to restore RIT 2023-07-12 15:10:21,322 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174651322 2023-07-12 15:10:21,322 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:10:21,323 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:10:21,325 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:21,325 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1eef801734bac1a9fe215d8599908306, regionState=OPENING, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:10:21,325 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:10:21,325 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174681325 2023-07-12 15:10:21,325 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174741325 2023-07-12 15:10:21,325 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:10:21,326 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:21,326 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174621325"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174621325"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174621325"}]},"ts":"1689174621325"} 2023-07-12 15:10:21,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:21,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1eef801734bac1a9fe215d8599908306, NAME => 'testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:21,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:21,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:21,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,490 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,491 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1 2023-07-12 15:10:21,491 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1 2023-07-12 15:10:21,492 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1eef801734bac1a9fe215d8599908306 columnFamilyName cf1 2023-07-12 15:10:21,499 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 2023-07-12 15:10:21,501 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 2023-07-12 15:10:21,506 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 2023-07-12 15:10:21,509 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 2023-07-12 15:10:21,511 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db 2023-07-12 15:10:21,514 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 2023-07-12 15:10:21,518 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 2023-07-12 15:10:21,518 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(310): Store=1eef801734bac1a9fe215d8599908306/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:21,518 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,519 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2 2023-07-12 15:10:21,519 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2 2023-07-12 15:10:21,519 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1eef801734bac1a9fe215d8599908306 columnFamilyName cf2 2023-07-12 15:10:21,526 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 2023-07-12 15:10:21,529 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a 2023-07-12 15:10:21,533 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd 2023-07-12 15:10:21,538 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 2023-07-12 15:10:21,543 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f 2023-07-12 15:10:21,546 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b 2023-07-12 15:10:21,549 DEBUG [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b 2023-07-12 15:10:21,550 INFO [StoreOpener-1eef801734bac1a9fe215d8599908306-1] regionserver.HStore(310): Store=1eef801734bac1a9fe215d8599908306/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:21,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.splits 2023-07-12 15:10:21,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:21,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1eef801734bac1a9fe215d8599908306; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64938060, jitterRate=-0.03234750032424927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:21,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:21,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306., pid=48, masterSystemTime=1689174621479 2023-07-12 15:10:21,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:21,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:21,557 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:21,557 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:21,558 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 15:10:21,558 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 15:10:21,558 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): 1eef801734bac1a9fe215d8599908306/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:21,558 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): 1eef801734bac1a9fe215d8599908306/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:21,558 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1eef801734bac1a9fe215d8599908306/cf2 in testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,559 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1eef801734bac1a9fe215d8599908306/cf1 in testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,559 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp, totalSize=34.9 K 2023-07-12 15:10:21,559 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp, totalSize=34.9 K 2023-07-12 15:10:21,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,560 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 50f3de1f42554b429917371f759fe52a, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174614541 2023-07-12 15:10:21,560 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 380a943e54cb457f8056f001d0c0a4e6, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174614541 2023-07-12 15:10:21,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:21,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174621560"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174621560"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174621560"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174621560"}]},"ts":"1689174621560"} 2023-07-12 15:10:21,560 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 56624e3549714c2db2521a397d6f07e8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174614714 2023-07-12 15:10:21,560 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 136d2acb5f2a41569acf48b43f2115b5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174614714 2023-07-12 15:10:21,560 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting f60323368a484d5ba92c05b5811df22b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174614906 2023-07-12 15:10:21,560 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 51c39dee82154f558039ce7a1fc7ca28, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174614906 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 82209269bda7498085a5baccf80cfb0f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174615110 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting db2ee8942a904e5ba4c8f832b6947f32, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174615110 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 9b526be88a1743169583c0a75e939c4b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174615304 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting f18976c407154b2591c4749ca57f59f7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174615304 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 1edcbef678234264858296a7273ae986, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174615512 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting ad005e70a51146f4aa620f8297c493db, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174615512 2023-07-12 15:10:21,561 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 56535eba61724ef183e575a8153a8dbd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174615706 2023-07-12 15:10:21,562 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 5075de4a740045acb884c03caa40f191, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174615706 2023-07-12 15:10:21,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-07-12 15:10:21,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 in 234 msec 2023-07-12 15:10:21,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN in 1.5710 sec 2023-07-12 15:10:21,576 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-07-12 15:10:21,581 DEBUG [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-07-12 15:10:21,582 INFO [Listener at localhost.localdomain/46471] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-12 15:10:21,589 INFO [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-07-12 15:10:21,589 INFO [Listener at localhost.localdomain/46471] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-12 15:10:21,589 INFO [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-07-12 15:10:21,594 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 1eef801734bac1a9fe215d8599908306#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:21,600 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1eef801734bac1a9fe215d8599908306#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:21,631 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf1/53386f36d3494e11ba34b6bd5595026c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/53386f36d3494e11ba34b6bd5595026c 2023-07-12 15:10:21,632 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/.tmp/cf2/30f355731acf484aa5b32954d84e6b27 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/30f355731acf484aa5b32954d84e6b27 2023-07-12 15:10:21,639 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1eef801734bac1a9fe215d8599908306/cf1 of 1eef801734bac1a9fe215d8599908306 into 53386f36d3494e11ba34b6bd5595026c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:21,639 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:21,639 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306., storeName=1eef801734bac1a9fe215d8599908306/cf1, priority=9, startTime=1689174621556; duration=0sec 2023-07-12 15:10:21,640 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:21,641 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1eef801734bac1a9fe215d8599908306/cf2 of 1eef801734bac1a9fe215d8599908306 into 30f355731acf484aa5b32954d84e6b27(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:21,641 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:21,641 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306., storeName=1eef801734bac1a9fe215d8599908306/cf2, priority=9, startTime=1689174621557; duration=0sec 2023-07-12 15:10:21,641 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:21,653 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:21,654 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-07-12 15:10:21,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRollbackAndDoubleExecution 2023-07-12 15:10:21,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-12 15:10:21,658 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174621658"}]},"ts":"1689174621658"} 2023-07-12 15:10:21,659 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-12 15:10:21,660 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-07-12 15:10:21,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN}] 2023-07-12 15:10:21,662 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN 2023-07-12 15:10:21,662 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:21,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174621662"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174621662"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174621662"}]},"ts":"1689174621662"} 2023-07-12 15:10:21,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:21,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-12 15:10:21,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1eef801734bac1a9fe215d8599908306, disabling compactions & flushes 2023-07-12 15:10:21,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. after waiting 0 ms 2023-07-12 15:10:21,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,819 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191] to archive 2023-07-12 15:10:21,821 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:21,824 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/380a943e54cb457f8056f001d0c0a4e6 2023-07-12 15:10:21,825 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/136d2acb5f2a41569acf48b43f2115b5 2023-07-12 15:10:21,827 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/51c39dee82154f558039ce7a1fc7ca28 2023-07-12 15:10:21,828 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/db2ee8942a904e5ba4c8f832b6947f32 2023-07-12 15:10:21,830 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/f18976c407154b2591c4749ca57f59f7 2023-07-12 15:10:21,831 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/ad005e70a51146f4aa620f8297c493db 2023-07-12 15:10:21,833 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/5075de4a740045acb884c03caa40f191 2023-07-12 15:10:21,838 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd] to archive 2023-07-12 15:10:21,839 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:21,840 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/50f3de1f42554b429917371f759fe52a 2023-07-12 15:10:21,841 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56624e3549714c2db2521a397d6f07e8 2023-07-12 15:10:21,842 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/f60323368a484d5ba92c05b5811df22b 2023-07-12 15:10:21,843 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/82209269bda7498085a5baccf80cfb0f 2023-07-12 15:10:21,844 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/9b526be88a1743169583c0a75e939c4b 2023-07-12 15:10:21,845 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/1edcbef678234264858296a7273ae986 2023-07-12 15:10:21,846 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/56535eba61724ef183e575a8153a8dbd 2023-07-12 15:10:21,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:10:21,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:21,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306. 2023-07-12 15:10:21,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1eef801734bac1a9fe215d8599908306: 2023-07-12 15:10:21,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,853 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=1eef801734bac1a9fe215d8599908306, regionState=CLOSED 2023-07-12 15:10:21,853 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174621853"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174621853"}]},"ts":"1689174621853"} 2023-07-12 15:10:21,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-07-12 15:10:21,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 1eef801734bac1a9fe215d8599908306, server=jenkins-hbase20.apache.org,37993,1689174599377 in 190 msec 2023-07-12 15:10:21,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-07-12 15:10:21,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, UNASSIGN in 196 msec 2023-07-12 15:10:21,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174621858"}]},"ts":"1689174621858"} 2023-07-12 15:10:21,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-12 15:10:21,860 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-07-12 15:10:21,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 206 msec 2023-07-12 15:10:21,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-12 15:10:21,963 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-07-12 15:10:21,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRollbackAndDoubleExecution 2023-07-12 15:10:21,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,966 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,967 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-12 15:10:21,970 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,971 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/recovered.edits] 2023-07-12 15:10:21,974 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/53386f36d3494e11ba34b6bd5595026c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf1/53386f36d3494e11ba34b6bd5595026c 2023-07-12 15:10:21,976 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/30f355731acf484aa5b32954d84e6b27 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/cf2/30f355731acf484aa5b32954d84e6b27 2023-07-12 15:10:21,978 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306/recovered.edits/90.seqid 2023-07-12 15:10:21,978 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackAndDoubleExecution/1eef801734bac1a9fe215d8599908306 2023-07-12 15:10:21,979 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-12 15:10:21,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,983 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-07-12 15:10:21,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-07-12 15:10:21,985 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,986 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-07-12 15:10:21,986 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174621986"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:21,987 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-12 15:10:21,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1eef801734bac1a9fe215d8599908306, NAME => 'testRollbackAndDoubleExecution,,1689174614033.1eef801734bac1a9fe215d8599908306.', STARTKEY => '', ENDKEY => ''}] 2023-07-12 15:10:21,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-07-12 15:10:21,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174621987"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:21,989 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-07-12 15:10:21,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:10:21,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 27 msec 2023-07-12 15:10:22,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-12 15:10:22,069 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-07-12 15:10:22,085 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=403 (was 407), OpenFileDescriptor=868 (was 870), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=347 (was 273) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 167) - ProcessCount LEAK? -, AvailableMemoryMB=5414 (was 5428) 2023-07-12 15:10:22,099 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=403, OpenFileDescriptor=868, MaxFileDescriptor=60000, SystemLoadAverage=347, ProcessCount=168, AvailableMemoryMB=5414 2023-07-12 15:10:22,099 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:22,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:10:22,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:10:22,104 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:22,105 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:22,106 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:22,108 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,108 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 empty. 2023-07-12 15:10:22,109 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,109 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-12 15:10:22,122 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:22,123 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 002a47310ffb17757afc3abd905a1bc6, NAME => 'testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:22,130 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:22,130 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 002a47310ffb17757afc3abd905a1bc6, disabling compactions & flushes 2023-07-12 15:10:22,130 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,130 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,130 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. after waiting 0 ms 2023-07-12 15:10:22,130 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,130 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,130 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 002a47310ffb17757afc3abd905a1bc6: 2023-07-12 15:10:22,132 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:22,133 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689174622133"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174622133"}]},"ts":"1689174622133"} 2023-07-12 15:10:22,134 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:22,135 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:22,136 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174622135"}]},"ts":"1689174622135"} 2023-07-12 15:10:22,137 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-07-12 15:10:22,139 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:22,140 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:22,140 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:22,140 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:22,140 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:22,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, ASSIGN}] 2023-07-12 15:10:22,142 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, ASSIGN 2023-07-12 15:10:22,143 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33703,1689174599345; forceNewPlan=false, retain=false 2023-07-12 15:10:22,293 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:22,296 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=002a47310ffb17757afc3abd905a1bc6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:22,297 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689174622296"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174622296"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174622296"}]},"ts":"1689174622296"} 2023-07-12 15:10:22,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 002a47310ffb17757afc3abd905a1bc6, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:22,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 002a47310ffb17757afc3abd905a1bc6, NAME => 'testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:22,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:22,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:22,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,471 INFO [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,474 DEBUG [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/cf1 2023-07-12 15:10:22,474 DEBUG [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/cf1 2023-07-12 15:10:22,475 INFO [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 002a47310ffb17757afc3abd905a1bc6 columnFamilyName cf1 2023-07-12 15:10:22,476 INFO [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] regionserver.HStore(310): Store=002a47310ffb17757afc3abd905a1bc6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:22,476 INFO [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,478 DEBUG [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/cf2 2023-07-12 15:10:22,478 DEBUG [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/cf2 2023-07-12 15:10:22,478 INFO [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 002a47310ffb17757afc3abd905a1bc6 columnFamilyName cf2 2023-07-12 15:10:22,479 INFO [StoreOpener-002a47310ffb17757afc3abd905a1bc6-1] regionserver.HStore(310): Store=002a47310ffb17757afc3abd905a1bc6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:22,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:22,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:22,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 002a47310ffb17757afc3abd905a1bc6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68280082, jitterRate=0.017452508211135864}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:22,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 002a47310ffb17757afc3abd905a1bc6: 2023-07-12 15:10:22,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6., pid=55, masterSystemTime=1689174622454 2023-07-12 15:10:22,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=002a47310ffb17757afc3abd905a1bc6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:22,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689174622489"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174622489"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174622489"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174622489"}]},"ts":"1689174622489"} 2023-07-12 15:10:22,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-07-12 15:10:22,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 002a47310ffb17757afc3abd905a1bc6, server=jenkins-hbase20.apache.org,33703,1689174599345 in 190 msec 2023-07-12 15:10:22,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-07-12 15:10:22,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, ASSIGN in 353 msec 2023-07-12 15:10:22,496 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:22,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174622496"}]},"ts":"1689174622496"} 2023-07-12 15:10:22,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-07-12 15:10:22,499 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:22,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 396 msec 2023-07-12 15:10:22,605 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:22,607 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:22,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:22,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:22,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:22,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, UNASSIGN}] 2023-07-12 15:10:22,614 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, UNASSIGN 2023-07-12 15:10:22,615 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=002a47310ffb17757afc3abd905a1bc6, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:22,615 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689174622615"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174622615"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174622615"}]},"ts":"1689174622615"} 2023-07-12 15:10:22,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 002a47310ffb17757afc3abd905a1bc6, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:22,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 002a47310ffb17757afc3abd905a1bc6, disabling compactions & flushes 2023-07-12 15:10:22,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. after waiting 0 ms 2023-07-12 15:10:22,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-12 15:10:22,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:22,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6. 2023-07-12 15:10:22,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 002a47310ffb17757afc3abd905a1bc6: 2023-07-12 15:10:22,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:22,786 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=002a47310ffb17757afc3abd905a1bc6, regionState=CLOSED 2023-07-12 15:10:22,786 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689174622786"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174622786"}]},"ts":"1689174622786"} 2023-07-12 15:10:22,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-07-12 15:10:22,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 002a47310ffb17757afc3abd905a1bc6, server=jenkins-hbase20.apache.org,33703,1689174599345 in 171 msec 2023-07-12 15:10:22,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-07-12 15:10:22,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=002a47310ffb17757afc3abd905a1bc6, UNASSIGN in 179 msec 2023-07-12 15:10:22,826 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-12 15:10:22,828 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-12 15:10:22,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1689174622829"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1689174622829"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1689174622829"}]},"ts":"1689174622829"} 2023-07-12 15:10:22,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174622829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174622829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174622829"}]},"ts":"1689174622829"} 2023-07-12 15:10:22,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174622829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174622829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174622829"}]},"ts":"1689174622829"} 2023-07-12 15:10:22,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, ASSIGN}] 2023-07-12 15:10:22,833 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, ASSIGN 2023-07-12 15:10:22,834 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33703,1689174599345; forceNewPlan=false, retain=false 2023-07-12 15:10:22,835 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, ASSIGN 2023-07-12 15:10:22,835 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,33703,1689174599345; forceNewPlan=false, retain=false 2023-07-12 15:10:22,984 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:10:22,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=420b5727eefe787777cba173bc4334d4, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:22,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174622986"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174622986"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174622986"}]},"ts":"1689174622986"} 2023-07-12 15:10:22,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 420b5727eefe787777cba173bc4334d4, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:22,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=db5251afa0b685e4e695d9600d5bcda1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:22,991 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174622990"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174622990"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174622990"}]},"ts":"1689174622990"} 2023-07-12 15:10:22,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure db5251afa0b685e4e695d9600d5bcda1, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:23,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 420b5727eefe787777cba173bc4334d4, NAME => 'testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 15:10:23,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:23,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:23,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,145 INFO [StoreOpener-420b5727eefe787777cba173bc4334d4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,146 DEBUG [StoreOpener-420b5727eefe787777cba173bc4334d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/cf1 2023-07-12 15:10:23,146 DEBUG [StoreOpener-420b5727eefe787777cba173bc4334d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/cf1 2023-07-12 15:10:23,147 INFO [StoreOpener-420b5727eefe787777cba173bc4334d4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 420b5727eefe787777cba173bc4334d4 columnFamilyName cf1 2023-07-12 15:10:23,148 INFO [StoreOpener-420b5727eefe787777cba173bc4334d4-1] regionserver.HStore(310): Store=420b5727eefe787777cba173bc4334d4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:23,148 INFO [StoreOpener-420b5727eefe787777cba173bc4334d4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,149 DEBUG [StoreOpener-420b5727eefe787777cba173bc4334d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/cf2 2023-07-12 15:10:23,149 DEBUG [StoreOpener-420b5727eefe787777cba173bc4334d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/cf2 2023-07-12 15:10:23,150 INFO [StoreOpener-420b5727eefe787777cba173bc4334d4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 420b5727eefe787777cba173bc4334d4 columnFamilyName cf2 2023-07-12 15:10:23,150 INFO [StoreOpener-420b5727eefe787777cba173bc4334d4-1] regionserver.HStore(310): Store=420b5727eefe787777cba173bc4334d4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:23,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:23,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 420b5727eefe787777cba173bc4334d4; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73185677, jitterRate=0.09055157005786896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:23,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 420b5727eefe787777cba173bc4334d4: 2023-07-12 15:10:23,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4., pid=61, masterSystemTime=1689174623140 2023-07-12 15:10:23,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => db5251afa0b685e4e695d9600d5bcda1, NAME => 'testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 15:10:23,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=420b5727eefe787777cba173bc4334d4, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:23,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:23,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174623160"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174623160"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174623160"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174623160"}]},"ts":"1689174623160"} 2023-07-12 15:10:23,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:23,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,163 INFO [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,164 DEBUG [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/cf1 2023-07-12 15:10:23,164 DEBUG [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/cf1 2023-07-12 15:10:23,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-07-12 15:10:23,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 420b5727eefe787777cba173bc4334d4, server=jenkins-hbase20.apache.org,33703,1689174599345 in 174 msec 2023-07-12 15:10:23,165 INFO [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db5251afa0b685e4e695d9600d5bcda1 columnFamilyName cf1 2023-07-12 15:10:23,165 INFO [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] regionserver.HStore(310): Store=db5251afa0b685e4e695d9600d5bcda1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:23,166 INFO [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, ASSIGN in 333 msec 2023-07-12 15:10:23,167 DEBUG [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/cf2 2023-07-12 15:10:23,167 DEBUG [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/cf2 2023-07-12 15:10:23,167 INFO [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db5251afa0b685e4e695d9600d5bcda1 columnFamilyName cf2 2023-07-12 15:10:23,167 INFO [StoreOpener-db5251afa0b685e4e695d9600d5bcda1-1] regionserver.HStore(310): Store=db5251afa0b685e4e695d9600d5bcda1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:23,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:23,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened db5251afa0b685e4e695d9600d5bcda1; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70621523, jitterRate=0.052342697978019714}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:23,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for db5251afa0b685e4e695d9600d5bcda1: 2023-07-12 15:10:23,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1., pid=62, masterSystemTime=1689174623140 2023-07-12 15:10:23,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,176 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=db5251afa0b685e4e695d9600d5bcda1, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:23,176 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174623176"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174623176"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174623176"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174623176"}]},"ts":"1689174623176"} 2023-07-12 15:10:23,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-07-12 15:10:23,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure db5251afa0b685e4e695d9600d5bcda1, server=jenkins-hbase20.apache.org,33703,1689174599345 in 185 msec 2023-07-12 15:10:23,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-07-12 15:10:23,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, ASSIGN in 347 msec 2023-07-12 15:10:23,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 in 576 msec 2023-07-12 15:10:23,367 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:10:23,374 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:23,376 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-12 15:10:23,381 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174623381"}]},"ts":"1689174623381"} 2023-07-12 15:10:23,383 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-07-12 15:10:23,384 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-07-12 15:10:23,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, UNASSIGN}] 2023-07-12 15:10:23,386 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, UNASSIGN 2023-07-12 15:10:23,387 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=420b5727eefe787777cba173bc4334d4, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:23,387 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174623387"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174623387"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174623387"}]},"ts":"1689174623387"} 2023-07-12 15:10:23,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 420b5727eefe787777cba173bc4334d4, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:23,393 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, UNASSIGN 2023-07-12 15:10:23,393 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=db5251afa0b685e4e695d9600d5bcda1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:23,394 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174623393"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174623393"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174623393"}]},"ts":"1689174623393"} 2023-07-12 15:10:23,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure db5251afa0b685e4e695d9600d5bcda1, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:23,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-12 15:10:23,542 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing db5251afa0b685e4e695d9600d5bcda1, disabling compactions & flushes 2023-07-12 15:10:23,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. after waiting 0 ms 2023-07-12 15:10:23,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-12 15:10:23,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:23,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1. 2023-07-12 15:10:23,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for db5251afa0b685e4e695d9600d5bcda1: 2023-07-12 15:10:23,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 420b5727eefe787777cba173bc4334d4, disabling compactions & flushes 2023-07-12 15:10:23,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. after waiting 0 ms 2023-07-12 15:10:23,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,556 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=db5251afa0b685e4e695d9600d5bcda1, regionState=CLOSED 2023-07-12 15:10:23,556 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174623556"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174623556"}]},"ts":"1689174623556"} 2023-07-12 15:10:23,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-07-12 15:10:23,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure db5251afa0b685e4e695d9600d5bcda1, server=jenkins-hbase20.apache.org,33703,1689174599345 in 162 msec 2023-07-12 15:10:23,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-12 15:10:23,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:23,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4. 2023-07-12 15:10:23,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 420b5727eefe787777cba173bc4334d4: 2023-07-12 15:10:23,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=db5251afa0b685e4e695d9600d5bcda1, UNASSIGN in 174 msec 2023-07-12 15:10:23,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=420b5727eefe787777cba173bc4334d4, regionState=CLOSED 2023-07-12 15:10:23,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174623562"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174623562"}]},"ts":"1689174623562"} 2023-07-12 15:10:23,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-07-12 15:10:23,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 420b5727eefe787777cba173bc4334d4, server=jenkins-hbase20.apache.org,33703,1689174599345 in 174 msec 2023-07-12 15:10:23,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-07-12 15:10:23,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=420b5727eefe787777cba173bc4334d4, UNASSIGN in 180 msec 2023-07-12 15:10:23,567 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174623567"}]},"ts":"1689174623567"} 2023-07-12 15:10:23,568 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-07-12 15:10:23,569 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-07-12 15:10:23,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 193 msec 2023-07-12 15:10:23,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-12 15:10:23,685 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-07-12 15:10:23,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,690 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,691 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-12 15:10:23,697 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:23,697 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,697 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,699 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/recovered.edits] 2023-07-12 15:10:23,699 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/recovered.edits] 2023-07-12 15:10:23,699 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/recovered.edits] 2023-07-12 15:10:23,705 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/recovered.edits/7.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4/recovered.edits/7.seqid 2023-07-12 15:10:23,705 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/recovered.edits/7.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1/recovered.edits/7.seqid 2023-07-12 15:10:23,705 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/recovered.edits/4.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6/recovered.edits/4.seqid 2023-07-12 15:10:23,705 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/420b5727eefe787777cba173bc4334d4 2023-07-12 15:10:23,706 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/db5251afa0b685e4e695d9600d5bcda1 2023-07-12 15:10:23,706 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionNoStoreFile/002a47310ffb17757afc3abd905a1bc6 2023-07-12 15:10:23,706 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-12 15:10:23,708 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,710 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-07-12 15:10:23,712 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-07-12 15:10:23,714 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,714 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-07-12 15:10:23,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174623714"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:23,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174623714"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:23,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174623714"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:23,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:10:23,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 002a47310ffb17757afc3abd905a1bc6, NAME => 'testSplitTableRegionNoStoreFile,,1689174622103.002a47310ffb17757afc3abd905a1bc6.', STARTKEY => '', ENDKEY => ''}, {ENCODED => db5251afa0b685e4e695d9600d5bcda1, NAME => 'testSplitTableRegionNoStoreFile,,1689174622605.db5251afa0b685e4e695d9600d5bcda1.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 420b5727eefe787777cba173bc4334d4, NAME => 'testSplitTableRegionNoStoreFile,41,1689174622605.420b5727eefe787777cba173bc4334d4.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 15:10:23,716 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-07-12 15:10:23,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174623716"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:23,718 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-07-12 15:10:23,719 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:10:23,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 33 msec 2023-07-12 15:10:23,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-12 15:10:23,793 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-07-12 15:10:23,814 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=410 (was 403) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-69066594_17 at /127.0.0.1:60622 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-69066594_17 at /127.0.0.1:42182 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 868), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=327 (was 347), ProcessCount=168 (was 168), AvailableMemoryMB=5413 (was 5414) 2023-07-12 15:10:23,830 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=410, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=327, ProcessCount=168, AvailableMemoryMB=5413 2023-07-12 15:10:23,831 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:23,833 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:10:23,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:10:23,835 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:23,837 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:23,837 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:23,839 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:23,839 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 empty. 2023-07-12 15:10:23,840 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:23,840 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-12 15:10:23,853 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:23,855 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => ba4d910ecaafdf5b7e02268c8df9ad23, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:23,870 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:23,870 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing ba4d910ecaafdf5b7e02268c8df9ad23, disabling compactions & flushes 2023-07-12 15:10:23,870 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:23,870 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:23,870 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. after waiting 0 ms 2023-07-12 15:10:23,870 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:23,870 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:23,870 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:23,872 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:23,873 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689174623873"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174623873"}]},"ts":"1689174623873"} 2023-07-12 15:10:23,874 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:23,874 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:23,874 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174623874"}]},"ts":"1689174623874"} 2023-07-12 15:10:23,875 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-07-12 15:10:23,877 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:23,878 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:23,878 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:23,878 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:23,878 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:23,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, ASSIGN}] 2023-07-12 15:10:23,880 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, ASSIGN 2023-07-12 15:10:23,880 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:24,031 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:24,033 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ba4d910ecaafdf5b7e02268c8df9ad23, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,034 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689174624033"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174624033"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174624033"}]},"ts":"1689174624033"} 2023-07-12 15:10:24,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure ba4d910ecaafdf5b7e02268c8df9ad23, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:24,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ba4d910ecaafdf5b7e02268c8df9ad23, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:24,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:24,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:24,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,196 INFO [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,197 DEBUG [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1 2023-07-12 15:10:24,197 DEBUG [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1 2023-07-12 15:10:24,198 INFO [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba4d910ecaafdf5b7e02268c8df9ad23 columnFamilyName cf1 2023-07-12 15:10:24,198 INFO [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] regionserver.HStore(310): Store=ba4d910ecaafdf5b7e02268c8df9ad23/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:24,198 INFO [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,199 DEBUG [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2 2023-07-12 15:10:24,199 DEBUG [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2 2023-07-12 15:10:24,200 INFO [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba4d910ecaafdf5b7e02268c8df9ad23 columnFamilyName cf2 2023-07-12 15:10:24,200 INFO [StoreOpener-ba4d910ecaafdf5b7e02268c8df9ad23-1] regionserver.HStore(310): Store=ba4d910ecaafdf5b7e02268c8df9ad23/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:24,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:24,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:24,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:24,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened ba4d910ecaafdf5b7e02268c8df9ad23; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62361913, jitterRate=-0.07073508203029633}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:24,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:24,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23., pid=71, masterSystemTime=1689174624192 2023-07-12 15:10:24,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,212 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ba4d910ecaafdf5b7e02268c8df9ad23, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,212 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689174624212"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174624212"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174624212"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174624212"}]},"ts":"1689174624212"} 2023-07-12 15:10:24,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-07-12 15:10:24,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure ba4d910ecaafdf5b7e02268c8df9ad23, server=jenkins-hbase20.apache.org,37993,1689174599377 in 175 msec 2023-07-12 15:10:24,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-07-12 15:10:24,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, ASSIGN in 337 msec 2023-07-12 15:10:24,217 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:24,218 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174624217"}]},"ts":"1689174624217"} 2023-07-12 15:10:24,219 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-07-12 15:10:24,220 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:24,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 386 msec 2023-07-12 15:10:24,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:24,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,344 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,344 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,344 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:24,344 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:24,345 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,345 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,346 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,346 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,346 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,346 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,346 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,346 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:24,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,347 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,348 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:24,348 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,348 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:24,349 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:10:24,349 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,350 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,350 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,350 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,350 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,350 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,365 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/a3ca3c9f281a4c58a7048844a1719553 2023-07-12 15:10:24,386 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/a5941ebd787b401d9fb5df2a5c7132bf 2023-07-12 15:10:24,391 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/a3ca3c9f281a4c58a7048844a1719553 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553 2023-07-12 15:10:24,395 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:24,396 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/a5941ebd787b401d9fb5df2a5c7132bf as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf 2023-07-12 15:10:24,400 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:10:24,400 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 51ms, sequenceid=6, compaction requested=false 2023-07-12 15:10:24,400 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,401 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:24,401 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,401 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:24,401 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:24,401 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,401 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,401 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,402 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,402 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,402 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,402 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:24,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dad469[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:24,404 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:24,404 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,404 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,404 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,404 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,405 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,405 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:24,405 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,406 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,406 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:24,409 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:24,409 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:24,409 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:24,409 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:24,409 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,409 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,410 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,410 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,410 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,411 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,412 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,412 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,412 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,416 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,416 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:24,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,417 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,517 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:24,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:24,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:24,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:24,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,538 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:24,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:24,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,540 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,540 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,540 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,540 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,540 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,540 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:24,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,541 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:24,542 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:24,542 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,542 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:24,542 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:24,543 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,543 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,543 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,543 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,544 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,544 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,557 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/b0be540f8ede4a8694967eb4cb92b65d 2023-07-12 15:10:24,574 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/497a427964fc498cbe7ba82677e69930 2023-07-12 15:10:24,579 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/b0be540f8ede4a8694967eb4cb92b65d as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d 2023-07-12 15:10:24,584 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:24,584 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/497a427964fc498cbe7ba82677e69930 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930 2023-07-12 15:10:24,589 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:10:24,590 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 48ms, sequenceid=19, compaction requested=false 2023-07-12 15:10:24,590 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:24,590 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,590 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:24,590 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:24,590 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,590 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,591 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,592 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,592 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,592 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,592 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:24,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7468c371[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:24,594 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:24,594 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,595 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:24,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:24,595 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,596 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,596 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,597 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:24,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:24,599 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:24,599 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:24,599 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:24,599 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,599 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:24,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,601 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,600 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,601 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,601 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,602 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,602 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,604 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,604 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,604 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:24,606 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:24,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,607 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,707 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:24,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:24,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:24,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:24,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,725 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,725 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,725 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:24,725 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:24,726 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,726 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,727 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,727 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,727 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,727 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,728 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,729 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:24,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:24,730 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,730 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:24,730 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:24,731 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,732 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,732 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,732 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,732 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,732 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,744 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/6bf32312b4bf4007b16c683506684f87 2023-07-12 15:10:24,755 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/1d1c839bce084c53947369a7fff22358 2023-07-12 15:10:24,759 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/6bf32312b4bf4007b16c683506684f87 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87 2023-07-12 15:10:24,762 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:24,763 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/1d1c839bce084c53947369a7fff22358 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358 2023-07-12 15:10:24,766 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:10:24,767 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 37ms, sequenceid=32, compaction requested=true 2023-07-12 15:10:24,767 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:24,767 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,767 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:24,767 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:24,768 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,768 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,768 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,769 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,769 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,769 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,769 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:24,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c32fa14[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:24,771 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:24,771 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,772 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:24,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:24,772 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:24,773 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,773 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,773 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,773 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,773 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,773 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,773 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,774 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:24,774 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,774 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:24,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:24,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:24,777 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:24,777 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:24,777 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:24,777 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,777 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,779 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:24,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,779 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,778 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,779 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,781 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,781 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:24,781 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,796 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,796 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,796 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,796 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,797 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:24,796 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:24,797 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,797 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:24,797 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:24,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,796 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,797 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,797 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:24,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:24,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,897 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:24,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:24,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:24,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:24,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,921 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:24,921 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,921 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:24,921 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:24,922 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,922 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,922 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,922 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,922 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,922 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:24,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:24,923 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:24,923 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:24,923 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,924 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,924 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,924 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:24,924 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,924 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:24,924 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,924 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:24,924 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:24,924 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:24,925 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,925 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,925 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,925 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,926 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:24,926 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:24,936 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/2fd845502eae44cdaa17880e0824e016 2023-07-12 15:10:25,350 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/a93bf06bb81a4b8ea7b4057765284531 2023-07-12 15:10:25,355 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/2fd845502eae44cdaa17880e0824e016 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016 2023-07-12 15:10:25,359 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:25,360 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/a93bf06bb81a4b8ea7b4057765284531 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531 2023-07-12 15:10:25,364 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:10:25,365 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 441ms, sequenceid=45, compaction requested=true 2023-07-12 15:10:25,365 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:25,365 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,366 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:25,366 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:25,366 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,366 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,366 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,367 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,367 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,367 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,367 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:25,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40ede58d[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:25,370 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:25,370 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,370 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,371 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,371 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,371 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:25,372 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,372 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,372 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,372 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,373 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,373 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,373 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,373 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,373 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,373 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:25,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:25,376 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:25,376 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:25,376 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:25,376 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,376 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,378 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,378 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,379 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,380 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,380 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,381 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,383 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,383 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,384 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,484 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:25,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:25,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:25,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:25,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,506 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,506 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,506 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:25,507 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:25,507 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,507 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,508 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,508 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,508 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,508 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,509 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,509 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:25,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,511 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,511 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,512 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,512 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,512 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,512 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,512 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,512 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,513 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,513 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,513 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,513 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,513 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,514 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:25,514 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:25,514 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,515 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:25,515 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:25,530 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/ad7844441a75468b9bc66a823260946e 2023-07-12 15:10:25,545 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/8cf49ac0d99544779bec91d11cd2893c 2023-07-12 15:10:25,549 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/ad7844441a75468b9bc66a823260946e as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e 2023-07-12 15:10:25,554 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:25,554 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/8cf49ac0d99544779bec91d11cd2893c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c 2023-07-12 15:10:25,559 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:10:25,560 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 45ms, sequenceid=58, compaction requested=true 2023-07-12 15:10:25,560 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:25,560 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,560 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:25,560 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:25,560 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,560 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,560 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,561 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,561 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,562 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,562 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:25,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15c10865[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:25,563 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:25,563 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,564 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,564 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,564 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,565 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,566 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,566 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,566 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,566 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,566 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:25,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:25,568 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:25,568 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:25,568 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:25,568 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,568 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,569 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,570 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,570 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,571 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,571 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,572 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:25,574 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,574 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:25,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:25,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,575 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,675 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:25,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:25,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:25,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:25,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,689 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,689 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,689 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:25,689 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:25,690 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,690 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,691 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,691 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,691 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,691 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,692 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:25,693 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,693 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:25,693 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:25,694 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:10:25,694 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,694 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,694 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,694 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,695 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,695 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,703 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/91ec52ad4c054c998791e93664eeb112 2023-07-12 15:10:25,718 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/835de3cbb41f456aafe8cf60cc22530b 2023-07-12 15:10:25,721 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/91ec52ad4c054c998791e93664eeb112 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112 2023-07-12 15:10:25,725 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:25,726 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/835de3cbb41f456aafe8cf60cc22530b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b 2023-07-12 15:10:25,729 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:10:25,730 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 37ms, sequenceid=71, compaction requested=true 2023-07-12 15:10:25,730 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:25,730 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:25,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:25,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,730 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,731 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,731 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,732 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,732 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:25,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@278032e7[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:25,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:25,733 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,734 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,734 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:25,735 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,736 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,736 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:25,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:25,740 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:25,740 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:25,740 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:25,740 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,740 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:25,749 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,749 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,749 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,749 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,749 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,749 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,750 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,750 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,750 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,751 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,752 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,752 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,753 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,757 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:25,758 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,759 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:25,759 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:25,758 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,759 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,759 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,760 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,760 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,760 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,859 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:25,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:25,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:25,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:25,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,885 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,885 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,885 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:25,885 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:25,885 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,885 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,886 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,886 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,886 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,886 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,886 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,886 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:25,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:25,887 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,887 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:25,888 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,888 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:25,888 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-07-12 15:10:25,888 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,889 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,889 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,889 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,889 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,889 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,896 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/0c3365a685f140dca478de964254bf78 2023-07-12 15:10:25,901 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c3365a685f140dca478de964254bf78 2023-07-12 15:10:25,911 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/bfb78404374449b598819e7c35457979 2023-07-12 15:10:25,917 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfb78404374449b598819e7c35457979 2023-07-12 15:10:25,918 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/0c3365a685f140dca478de964254bf78 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78 2023-07-12 15:10:25,923 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c3365a685f140dca478de964254bf78 2023-07-12 15:10:25,923 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78, entries=9, sequenceid=83, filesize=5.1 K 2023-07-12 15:10:25,924 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/bfb78404374449b598819e7c35457979 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979 2023-07-12 15:10:25,928 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfb78404374449b598819e7c35457979 2023-07-12 15:10:25,928 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979, entries=9, sequenceid=83, filesize=5.1 K 2023-07-12 15:10:25,929 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 41ms, sequenceid=83, compaction requested=true 2023-07-12 15:10:25,929 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:25,929 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:25,929 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:25,929 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:25,929 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,929 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:25,929 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:25,931 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,931 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,931 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,931 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:25,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cf98e59[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:25,933 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:25,933 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,934 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:25,934 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:25,936 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,936 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,936 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,937 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:25,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:25,941 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:25,941 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:25,941 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:25,941 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,941 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:25,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,950 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:25,950 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,951 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:25,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,952 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:25,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,953 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,954 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,954 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:25,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:25,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:25,957 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,958 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:25,958 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,958 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:25,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:25,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:25,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:25,958 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:25,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:25,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:25,959 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:25,959 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,058 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:26,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:26,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:26,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:26,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,078 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:26,078 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,078 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:26,078 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:26,079 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,079 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,080 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,080 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,080 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:26,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:26,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,081 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:26,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:26,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:26,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:26,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:26,082 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,082 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:26,082 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-12 15:10:26,082 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,083 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,083 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,083 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:26,083 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,083 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:26,091 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:26,095 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:26,106 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:26,110 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:26,111 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:26,115 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:26,115 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8, entries=5, sequenceid=91, filesize=5.0 K 2023-07-12 15:10:26,116 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/afafef5de87d434bac62fe80fc948a2c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:26,120 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:26,120 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c, entries=5, sequenceid=91, filesize=5.0 K 2023-07-12 15:10:26,121 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 39ms, sequenceid=91, compaction requested=true 2023-07-12 15:10:26,121 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:26,121 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,121 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:26,121 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:26,121 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,122 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:26,122 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:26,123 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,123 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:26,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:26,123 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,124 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:26,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:26,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:26,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:26,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:26,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1317d5c6[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:26,126 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:26,126 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:26,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:26,127 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,128 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:26,128 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:26,128 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:26,127 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:26,128 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:26,128 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:26,129 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,129 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:26,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:26,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:26,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:26,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:26,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:26,133 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:26,133 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:26,133 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:26,133 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,133 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,134 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,135 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,135 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:26,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,136 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,138 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,138 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,138 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,150 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,150 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,150 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,150 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,150 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,150 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:26,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:26,151 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,152 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:26,152 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:26,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,153 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,153 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,153 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,153 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,153 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,252 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:26,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:26,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:10:26,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:10:26,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,272 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:26,272 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,272 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-12 15:10:26,272 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-12 15:10:26,273 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,273 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,274 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,274 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,274 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,274 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,274 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,274 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:10:26,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,276 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,275 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,276 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:26,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:26,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-12 15:10:26,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:26,276 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-12 15:10:26,277 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:26,277 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:26,277 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:10:26,277 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:10:26,277 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,277 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. started... 2023-07-12 15:10:26,277 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-12 15:10:26,278 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,278 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,278 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:26,278 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,279 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,279 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:26,289 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:26,294 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:26,305 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:26,309 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:26,310 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:26,314 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:26,314 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7, entries=5, sequenceid=99, filesize=5.0 K 2023-07-12 15:10:26,315 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/46403158bcca4a4c9b19920114df55ad as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:26,321 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:26,321 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad, entries=5, sequenceid=99, filesize=5.0 K 2023-07-12 15:10:26,322 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 45ms, sequenceid=99, compaction requested=true 2023-07-12 15:10:26,322 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:26,322 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:10:26,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:10:26,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-12 15:10:26,322 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-12 15:10:26,323 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,323 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:26,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:26,324 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,324 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:10:26,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:26,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:26,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:26,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-12 15:10:26,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e8865e1[Count = 0] remaining members to acquire global barrier 2023-07-12 15:10:26,326 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-12 15:10:26,326 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:26,328 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,328 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-12 15:10:26,328 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:26,329 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:26,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:26,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-12 15:10:26,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:26,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:26,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:10:26,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:10:26,334 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-12 15:10:26,334 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:10:26,334 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:10:26,334 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,334 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:10:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:10:26,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,341 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,342 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:10:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,343 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,344 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:10:26,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:10:26,346 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,347 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,348 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:10:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,353 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,352 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-12 15:10:26,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:10:26,352 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:10:26,353 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:10:26,353 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:10:26,353 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:10:26,353 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:10:26,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:10:26,353 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,354 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,354 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:26,354 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,354 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:26,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:10:26,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:10:26,454 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:10:26,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:10:26,458 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:26,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:26,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:26,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:26,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, UNASSIGN}] 2023-07-12 15:10:26,465 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, UNASSIGN 2023-07-12 15:10:26,466 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ba4d910ecaafdf5b7e02268c8df9ad23, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:26,466 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689174626466"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174626466"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174626466"}]},"ts":"1689174626466"} 2023-07-12 15:10:26,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure ba4d910ecaafdf5b7e02268c8df9ad23, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:26,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing ba4d910ecaafdf5b7e02268c8df9ad23, disabling compactions & flushes 2023-07-12 15:10:26,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. after waiting 0 ms 2023-07-12 15:10:26,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing ba4d910ecaafdf5b7e02268c8df9ad23 2/2 column families, dataSize=50 B heapSize=720 B 2023-07-12 15:10:26,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:26,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:26,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:26,660 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:26,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf1/ac1a931fcfbe48ed94ec508b61174f6f as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:26,665 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:26,665 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f, entries=1, sequenceid=103, filesize=4.9 K 2023-07-12 15:10:26,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.tmp/cf2/d9bd15a6ad354590bf9c6b742a08d890 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:26,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:26,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890, entries=1, sequenceid=103, filesize=4.9 K 2023-07-12 15:10:26,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for ba4d910ecaafdf5b7e02268c8df9ad23 in 53ms, sequenceid=103, compaction requested=true 2023-07-12 15:10:26,679 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-07-12 15:10:26,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:26,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23. 2023-07-12 15:10:26,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for ba4d910ecaafdf5b7e02268c8df9ad23: 2023-07-12 15:10:26,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,682 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ba4d910ecaafdf5b7e02268c8df9ad23, regionState=CLOSED 2023-07-12 15:10:26,682 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689174626682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174626682"}]},"ts":"1689174626682"} 2023-07-12 15:10:26,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-07-12 15:10:26,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure ba4d910ecaafdf5b7e02268c8df9ad23, server=jenkins-hbase20.apache.org,37993,1689174599377 in 216 msec 2023-07-12 15:10:26,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-07-12 15:10:26,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ba4d910ecaafdf5b7e02268c8df9ad23, UNASSIGN in 222 msec 2023-07-12 15:10:26,700 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=ba4d910ecaafdf5b7e02268c8df9ad23, threads=16 2023-07-12 15:10:26,701 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,701 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,701 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,702 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,706 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,707 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,707 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,707 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,707 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,707 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,707 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,708 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,708 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,708 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,708 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,708 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,712 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfb78404374449b598819e7c35457979 2023-07-12 15:10:26,713 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531, top=false 2023-07-12 15:10:26,713 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b, top=false 2023-07-12 15:10:26,715 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c, top=false 2023-07-12 15:10:26,716 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:26,721 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:26,721 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112, top=false 2023-07-12 15:10:26,721 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c3365a685f140dca478de964254bf78 2023-07-12 15:10:26,722 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:26,722 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad, top=true 2023-07-12 15:10:26,722 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:26,735 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:26,735 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016, top=false 2023-07-12 15:10:26,759 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:26,759 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c, top=true 2023-07-12 15:10:26,762 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:26,762 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8, top=true 2023-07-12 15:10:26,763 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b for child: 32a3e04fd3a4ef4f387a0d2b230ceb89, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,771 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531 for child: 32a3e04fd3a4ef4f387a0d2b230ceb89, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,778 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,778 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,779 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:26,779 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890, top=true 2023-07-12 15:10:26,779 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,779 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,780 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfb78404374449b598819e7c35457979 2023-07-12 15:10:26,783 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:26,783 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e, top=false 2023-07-12 15:10:26,786 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:26,786 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f, top=true 2023-07-12 15:10:26,792 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112 for child: 32a3e04fd3a4ef4f387a0d2b230ceb89, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,794 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c for child: 32a3e04fd3a4ef4f387a0d2b230ceb89, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,795 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,795 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,797 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,797 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,800 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:26,803 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:26,803 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7, top=true 2023-07-12 15:10:26,815 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad for child: ad57113446f90ac0969654c5c8db488f, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,815 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,836 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c for child: ad57113446f90ac0969654c5c8db488f, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,836 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,839 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c3365a685f140dca478de964254bf78 2023-07-12 15:10:26,842 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e for child: 32a3e04fd3a4ef4f387a0d2b230ceb89, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,843 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8 for child: ad57113446f90ac0969654c5c8db488f, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,843 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016 for child: 32a3e04fd3a4ef4f387a0d2b230ceb89, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,843 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,846 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,846 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,848 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f for child: ad57113446f90ac0969654c5c8db488f, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,848 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890 for child: ad57113446f90ac0969654c5c8db488f, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,848 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,848 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,848 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/.splits/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7 for child: ad57113446f90ac0969654c5c8db488f, parent: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,848 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,856 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,857 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,867 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,875 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,878 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,879 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930 for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,888 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,889 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d for region: ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:26,889 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region ba4d910ecaafdf5b7e02268c8df9ad23 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:10:26,929 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-12 15:10:26,932 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-12 15:10:26,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1689174626933"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1689174626933"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1689174626933"}]},"ts":"1689174626933"} 2023-07-12 15:10:26,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174626933"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174626933"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174626933"}]},"ts":"1689174626933"} 2023-07-12 15:10:26,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174626933"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174626933"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174626933"}]},"ts":"1689174626933"} 2023-07-12 15:10:26,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, ASSIGN}] 2023-07-12 15:10:26,939 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, ASSIGN 2023-07-12 15:10:26,940 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:26,941 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, ASSIGN 2023-07-12 15:10:26,942 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:10:26,992 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:10:27,090 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:10:27,092 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=ad57113446f90ac0969654c5c8db488f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:27,092 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627092"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174627092"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174627092"}]},"ts":"1689174627092"} 2023-07-12 15:10:27,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure ad57113446f90ac0969654c5c8db488f, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:27,095 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=32a3e04fd3a4ef4f387a0d2b230ceb89, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:27,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627095"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174627095"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174627095"}]},"ts":"1689174627095"} 2023-07-12 15:10:27,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 32a3e04fd3a4ef4f387a0d2b230ceb89, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:27,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ad57113446f90ac0969654c5c8db488f, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.', STARTKEY => '60', ENDKEY => ''} 2023-07-12 15:10:27,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:27,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:27,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,257 INFO [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,258 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1 2023-07-12 15:10:27,258 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1 2023-07-12 15:10:27,259 INFO [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad57113446f90ac0969654c5c8db488f columnFamilyName cf1 2023-07-12 15:10:27,273 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,274 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78-top 2023-07-12 15:10:27,278 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87-top 2023-07-12 15:10:27,282 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553-top 2023-07-12 15:10:27,286 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d-top 2023-07-12 15:10:27,289 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:27,289 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:27,293 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:27,293 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:27,296 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:27,296 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:27,296 INFO [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(310): Store=ad57113446f90ac0969654c5c8db488f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:27,296 INFO [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,297 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2 2023-07-12 15:10:27,297 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2 2023-07-12 15:10:27,298 INFO [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad57113446f90ac0969654c5c8db488f columnFamilyName cf2 2023-07-12 15:10:27,307 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358-top 2023-07-12 15:10:27,312 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930-top 2023-07-12 15:10:27,316 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf-top 2023-07-12 15:10:27,320 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,321 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979-top 2023-07-12 15:10:27,324 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:27,324 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:27,327 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:27,327 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:27,331 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:27,331 DEBUG [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:27,331 INFO [StoreOpener-ad57113446f90ac0969654c5c8db488f-1] regionserver.HStore(310): Store=ad57113446f90ac0969654c5c8db488f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:27,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:27,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened ad57113446f90ac0969654c5c8db488f; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64222424, jitterRate=-0.04301130771636963}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:27,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for ad57113446f90ac0969654c5c8db488f: 2023-07-12 15:10:27,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f., pid=77, masterSystemTime=1689174627244 2023-07-12 15:10:27,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,337 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:27,337 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:27,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,337 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,337 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,338 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): ad57113446f90ac0969654c5c8db488f/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:27,338 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): ad57113446f90ac0969654c5c8db488f/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:27,338 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad57113446f90ac0969654c5c8db488f/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,338 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ad57113446f90ac0969654c5c8db488f/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,338 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp, totalSize=34.8 K 2023-07-12 15:10:27,338 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp, totalSize=34.8 K 2023-07-12 15:10:27,339 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174624338 2023-07-12 15:10:27,339 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174624338 2023-07-12 15:10:27,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,339 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174624519 2023-07-12 15:10:27,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 32a3e04fd3a4ef4f387a0d2b230ceb89, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.', STARTKEY => '', ENDKEY => '60'} 2023-07-12 15:10:27,340 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=ad57113446f90ac0969654c5c8db488f, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:27,340 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627340"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174627340"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174627340"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174627340"}]},"ts":"1689174627340"} 2023-07-12 15:10:27,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:27,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:27,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,341 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174624519 2023-07-12 15:10:27,341 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174624709 2023-07-12 15:10:27,341 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174624709 2023-07-12 15:10:27,341 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1689174625864 2023-07-12 15:10:27,342 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1689174625864 2023-07-12 15:10:27,342 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,342 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,342 INFO [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,343 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,343 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,343 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1 2023-07-12 15:10:27,343 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-07-12 15:10:27,344 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,343 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1 2023-07-12 15:10:27,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure ad57113446f90ac0969654c5c8db488f, server=jenkins-hbase20.apache.org,37993,1689174599377 in 248 msec 2023-07-12 15:10:27,344 INFO [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 32a3e04fd3a4ef4f387a0d2b230ceb89 columnFamilyName cf1 2023-07-12 15:10:27,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, ASSIGN in 406 msec 2023-07-12 15:10:27,356 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,357 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78-bottom 2023-07-12 15:10:27,357 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad57113446f90ac0969654c5c8db488f#cf1#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,363 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): ad57113446f90ac0969654c5c8db488f#cf2#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,365 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87-bottom 2023-07-12 15:10:27,369 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553-bottom 2023-07-12 15:10:27,372 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d-bottom 2023-07-12 15:10:27,376 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016 2023-07-12 15:10:27,381 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112 2023-07-12 15:10:27,383 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e 2023-07-12 15:10:27,387 INFO [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(310): Store=32a3e04fd3a4ef4f387a0d2b230ceb89/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:27,387 INFO [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,388 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 2023-07-12 15:10:27,388 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 2023-07-12 15:10:27,388 INFO [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 32a3e04fd3a4ef4f387a0d2b230ceb89 columnFamilyName cf2 2023-07-12 15:10:27,398 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for dc33b02026b24bdc9e59d19010bdc37e: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,399 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp/cf1/dc33b02026b24bdc9e59d19010bdc37e as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/dc33b02026b24bdc9e59d19010bdc37e 2023-07-12 15:10:27,403 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for dc33b02026b24bdc9e59d19010bdc37e: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,405 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ad57113446f90ac0969654c5c8db488f/cf1 of ad57113446f90ac0969654c5c8db488f into dc33b02026b24bdc9e59d19010bdc37e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,405 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad57113446f90ac0969654c5c8db488f: 2023-07-12 15:10:27,405 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f., storeName=ad57113446f90ac0969654c5c8db488f/cf1, priority=9, startTime=1689174627336; duration=0sec 2023-07-12 15:10:27,405 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,405 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358-bottom 2023-07-12 15:10:27,408 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4bce502cfc3d4a27a3d50803eac8359c: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,408 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930-bottom 2023-07-12 15:10:27,409 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp/cf2/4bce502cfc3d4a27a3d50803eac8359c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/4bce502cfc3d4a27a3d50803eac8359c 2023-07-12 15:10:27,412 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf-bottom 2023-07-12 15:10:27,413 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4bce502cfc3d4a27a3d50803eac8359c: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,414 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ad57113446f90ac0969654c5c8db488f/cf2 of ad57113446f90ac0969654c5c8db488f into 4bce502cfc3d4a27a3d50803eac8359c(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,414 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad57113446f90ac0969654c5c8db488f: 2023-07-12 15:10:27,414 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f., storeName=ad57113446f90ac0969654c5c8db488f/cf2, priority=9, startTime=1689174627337; duration=0sec 2023-07-12 15:10:27,414 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,416 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,416 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979-bottom 2023-07-12 15:10:27,420 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b 2023-07-12 15:10:27,423 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c 2023-07-12 15:10:27,427 DEBUG [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531 2023-07-12 15:10:27,427 INFO [StoreOpener-32a3e04fd3a4ef4f387a0d2b230ceb89-1] regionserver.HStore(310): Store=32a3e04fd3a4ef4f387a0d2b230ceb89/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:27,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,430 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:10:27,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 32a3e04fd3a4ef4f387a0d2b230ceb89; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60548589, jitterRate=-0.09775571525096893}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:10:27,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 32a3e04fd3a4ef4f387a0d2b230ceb89: 2023-07-12 15:10:27,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89., pid=78, masterSystemTime=1689174627244 2023-07-12 15:10:27,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,434 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:27,434 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:10:27,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:27,434 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,434 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,434 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): 32a3e04fd3a4ef4f387a0d2b230ceb89/cf1 is initiating minor compaction (all files) 2023-07-12 15:10:27,434 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): 32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 is initiating minor compaction (all files) 2023-07-12 15:10:27,434 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 32a3e04fd3a4ef4f387a0d2b230ceb89/cf1 in testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,434 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 in testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,435 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/.tmp, totalSize=35.0 K 2023-07-12 15:10:27,435 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/.tmp, totalSize=35.0 K 2023-07-12 15:10:27,435 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174624338 2023-07-12 15:10:27,435 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174624338 2023-07-12 15:10:27,435 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174624519 2023-07-12 15:10:27,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,436 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174624709 2023-07-12 15:10:27,436 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174624519 2023-07-12 15:10:27,436 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=32a3e04fd3a4ef4f387a0d2b230ceb89, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:27,436 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627436"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174627436"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174627436"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174627436"}]},"ts":"1689174627436"} 2023-07-12 15:10:27,436 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174624900 2023-07-12 15:10:27,436 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174624709 2023-07-12 15:10:27,436 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174625487 2023-07-12 15:10:27,437 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174624900 2023-07-12 15:10:27,437 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174625677 2023-07-12 15:10:27,437 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174625864 2023-07-12 15:10:27,437 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174625487 2023-07-12 15:10:27,438 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174625677 2023-07-12 15:10:27,438 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174625864 2023-07-12 15:10:27,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-07-12 15:10:27,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 32a3e04fd3a4ef4f387a0d2b230ceb89, server=jenkins-hbase20.apache.org,37993,1689174599377 in 341 msec 2023-07-12 15:10:27,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-07-12 15:10:27,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, ASSIGN in 501 msec 2023-07-12 15:10:27,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f in 982 msec 2023-07-12 15:10:27,449 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 32a3e04fd3a4ef4f387a0d2b230ceb89#cf2#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,451 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): 32a3e04fd3a4ef4f387a0d2b230ceb89#cf1#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,468 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/.tmp/cf2/eba596dde11b40588a0f8796b0b90397 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/eba596dde11b40588a0f8796b0b90397 2023-07-12 15:10:27,471 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/.tmp/cf1/5dd46ccb27744bf7aaaf110ef93b4918 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/5dd46ccb27744bf7aaaf110ef93b4918 2023-07-12 15:10:27,473 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 of 32a3e04fd3a4ef4f387a0d2b230ceb89 into eba596dde11b40588a0f8796b0b90397(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,473 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 32a3e04fd3a4ef4f387a0d2b230ceb89: 2023-07-12 15:10:27,473 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89., storeName=32a3e04fd3a4ef4f387a0d2b230ceb89/cf2, priority=9, startTime=1689174627434; duration=0sec 2023-07-12 15:10:27,473 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,475 DEBUG [Listener at localhost.localdomain/46471] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-12 15:10:27,479 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:60926, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-12 15:10:27,479 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. because compaction request was cancelled 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] regionserver.HStore(1912): 32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 is initiating major compaction (all files) 2023-07-12 15:10:27,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4a9b170d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,480 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 in testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,481 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/eba596dde11b40588a0f8796b0b90397] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/.tmp, totalSize=6.8 K 2023-07-12 15:10:27,481 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting eba596dde11b40588a0f8796b0b90397, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174624338 2023-07-12 15:10:27,482 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 32a3e04fd3a4ef4f387a0d2b230ceb89/cf1 of 32a3e04fd3a4ef4f387a0d2b230ceb89 into 5dd46ccb27744bf7aaaf110ef93b4918(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,482 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 32a3e04fd3a4ef4f387a0d2b230ceb89: 2023-07-12 15:10:27,482 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89., storeName=32a3e04fd3a4ef4f387a0d2b230ceb89/cf1, priority=9, startTime=1689174627433; duration=0sec 2023-07-12 15:10:27,483 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,483 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-12 15:10:27,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.HStore(1912): ad57113446f90ac0969654c5c8db488f/cf1 is initiating major compaction (all files) 2023-07-12 15:10:27,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@68d30923; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,483 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad57113446f90ac0969654c5c8db488f/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-12 15:10:27,483 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/dc33b02026b24bdc9e59d19010bdc37e] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp, totalSize=5.0 K 2023-07-12 15:10:27,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.HStore(1912): ad57113446f90ac0969654c5c8db488f/cf2 is initiating major compaction (all files) 2023-07-12 15:10:27,483 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@48ac02de; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:27,483 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting dc33b02026b24bdc9e59d19010bdc37e, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,484 INFO [Listener at localhost.localdomain/46471] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-07-12 15:10:27,485 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 32a3e04fd3a4ef4f387a0d2b230ceb89#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,486 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad57113446f90ac0969654c5c8db488f#cf1#compaction#77 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,500 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. is not mergeable because it has references 2023-07-12 15:10:27,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. is not mergeable because it has references 2023-07-12 15:10:27,506 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0e400b85cba04355a8415817330b41ce: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,506 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/.tmp/cf2/dfa43a90c84648ae84c199cff33c8c94 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/dfa43a90c84648ae84c199cff33c8c94 2023-07-12 15:10:27,506 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp/cf1/0e400b85cba04355a8415817330b41ce as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0e400b85cba04355a8415817330b41ce 2023-07-12 15:10:27,510 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0e400b85cba04355a8415817330b41ce: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,511 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 32a3e04fd3a4ef4f387a0d2b230ceb89/cf2 of 32a3e04fd3a4ef4f387a0d2b230ceb89 into dfa43a90c84648ae84c199cff33c8c94(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,511 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in ad57113446f90ac0969654c5c8db488f/cf1 of ad57113446f90ac0969654c5c8db488f into 0e400b85cba04355a8415817330b41ce(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,511 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 32a3e04fd3a4ef4f387a0d2b230ceb89: 2023-07-12 15:10:27,511 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad57113446f90ac0969654c5c8db488f: 2023-07-12 15:10:27,512 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1689174627480; duration=0sec 2023-07-12 15:10:27,512 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1689174627483; duration=0sec 2023-07-12 15:10:27,513 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:10:27,513 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,513 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ad57113446f90ac0969654c5c8db488f/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,513 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/4bce502cfc3d4a27a3d50803eac8359c] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp, totalSize=5.0 K 2023-07-12 15:10:27,513 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 4bce502cfc3d4a27a3d50803eac8359c, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-12 15:10:27,516 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): ad57113446f90ac0969654c5c8db488f#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:10:27,530 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 584e2d7815f14b7e8dfb9d2a94674bbc: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,530 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/.tmp/cf2/584e2d7815f14b7e8dfb9d2a94674bbc as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/584e2d7815f14b7e8dfb9d2a94674bbc 2023-07-12 15:10:27,534 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 584e2d7815f14b7e8dfb9d2a94674bbc: NONE, but ROW specified in column family configuration 2023-07-12 15:10:27,535 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in ad57113446f90ac0969654c5c8db488f/cf2 of ad57113446f90ac0969654c5c8db488f into 584e2d7815f14b7e8dfb9d2a94674bbc(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:10:27,535 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad57113446f90ac0969654c5c8db488f: 2023-07-12 15:10:27,535 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1689174627483; duration=0sec 2023-07-12 15:10:27,535 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:10:27,610 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37993] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. is not mergeable because it has references 2023-07-12 15:10:27,611 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37993] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. is not mergeable because it has references 2023-07-12 15:10:27,619 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-07-12 15:10:27,621 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:27,623 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-12 15:10:27,629 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174627629"}]},"ts":"1689174627629"} 2023-07-12 15:10:27,630 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-07-12 15:10:27,631 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-07-12 15:10:27,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, UNASSIGN}] 2023-07-12 15:10:27,633 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, UNASSIGN 2023-07-12 15:10:27,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ad57113446f90ac0969654c5c8db488f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:27,634 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627634"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174627634"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174627634"}]},"ts":"1689174627634"} 2023-07-12 15:10:27,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure ad57113446f90ac0969654c5c8db488f, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:27,638 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, UNASSIGN 2023-07-12 15:10:27,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=32a3e04fd3a4ef4f387a0d2b230ceb89, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:10:27,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627638"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174627638"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174627638"}]},"ts":"1689174627638"} 2023-07-12 15:10:27,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 32a3e04fd3a4ef4f387a0d2b230ceb89, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:10:27,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-12 15:10:27,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 32a3e04fd3a4ef4f387a0d2b230ceb89, disabling compactions & flushes 2023-07-12 15:10:27,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. after waiting 0 ms 2023-07-12 15:10:27,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,790 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78-bottom] to archive 2023-07-12 15:10:27,792 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:27,796 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,799 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,801 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,804 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-2fd845502eae44cdaa17880e0824e016 2023-07-12 15:10:27,805 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ad7844441a75468b9bc66a823260946e 2023-07-12 15:10:27,807 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-91ec52ad4c054c998791e93664eeb112 2023-07-12 15:10:27,809 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,811 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/eba596dde11b40588a0f8796b0b90397, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979-bottom] to archive 2023-07-12 15:10:27,812 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:27,815 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,816 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,818 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,819 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-a93bf06bb81a4b8ea7b4057765284531 2023-07-12 15:10:27,820 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-8cf49ac0d99544779bec91d11cd2893c 2023-07-12 15:10:27,821 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-835de3cbb41f456aafe8cf60cc22530b 2023-07-12 15:10:27,822 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/eba596dde11b40588a0f8796b0b90397 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/eba596dde11b40588a0f8796b0b90397 2023-07-12 15:10:27,823 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/recovered.edits/112.seqid, newMaxSeqId=112, maxSeqId=106 2023-07-12 15:10:27,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:27,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89. 2023-07-12 15:10:27,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 32a3e04fd3a4ef4f387a0d2b230ceb89: 2023-07-12 15:10:27,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing ad57113446f90ac0969654c5c8db488f, disabling compactions & flushes 2023-07-12 15:10:27,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. after waiting 0 ms 2023-07-12 15:10:27,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=32a3e04fd3a4ef4f387a0d2b230ceb89, regionState=CLOSED 2023-07-12 15:10:27,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,829 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174627829"}]},"ts":"1689174627829"} 2023-07-12 15:10:27,831 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/dc33b02026b24bdc9e59d19010bdc37e, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f] to archive 2023-07-12 15:10:27,831 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:27,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-07-12 15:10:27,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 32a3e04fd3a4ef4f387a0d2b230ceb89, server=jenkins-hbase20.apache.org,37993,1689174599377 in 191 msec 2023-07-12 15:10:27,833 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/a3ca3c9f281a4c58a7048844a1719553.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=32a3e04fd3a4ef4f387a0d2b230ceb89, UNASSIGN in 200 msec 2023-07-12 15:10:27,834 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/b0be540f8ede4a8694967eb4cb92b65d.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,835 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/6bf32312b4bf4007b16c683506684f87.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,836 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0c3365a685f140dca478de964254bf78.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,836 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:27,837 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:27,838 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/dc33b02026b24bdc9e59d19010bdc37e to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/dc33b02026b24bdc9e59d19010bdc37e 2023-07-12 15:10:27,839 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:27,841 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/4bce502cfc3d4a27a3d50803eac8359c, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890] to archive 2023-07-12 15:10:27,841 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:10:27,843 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/a5941ebd787b401d9fb5df2a5c7132bf.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,844 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/497a427964fc498cbe7ba82677e69930.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,845 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/1d1c839bce084c53947369a7fff22358.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,845 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/bfb78404374449b598819e7c35457979.ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,846 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:27,847 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:27,848 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/4bce502cfc3d4a27a3d50803eac8359c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/4bce502cfc3d4a27a3d50803eac8359c 2023-07-12 15:10:27,849 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/testSplitTableRegionDeletedRowsDaughter=ba4d910ecaafdf5b7e02268c8df9ad23-d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:27,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-07-12 15:10:27,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:27,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f. 2023-07-12 15:10:27,853 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for ad57113446f90ac0969654c5c8db488f: 2023-07-12 15:10:27,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,854 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ad57113446f90ac0969654c5c8db488f, regionState=CLOSED 2023-07-12 15:10:27,854 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689174627854"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174627854"}]},"ts":"1689174627854"} 2023-07-12 15:10:27,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-07-12 15:10:27,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure ad57113446f90ac0969654c5c8db488f, server=jenkins-hbase20.apache.org,37993,1689174599377 in 220 msec 2023-07-12 15:10:27,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-07-12 15:10:27,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ad57113446f90ac0969654c5c8db488f, UNASSIGN in 225 msec 2023-07-12 15:10:27,859 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174627859"}]},"ts":"1689174627859"} 2023-07-12 15:10:27,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-07-12 15:10:27,861 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-07-12 15:10:27,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 237 msec 2023-07-12 15:10:27,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-12 15:10:27,932 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-07-12 15:10:27,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-12 15:10:27,947 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,947 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,947 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,950 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/recovered.edits] 2023-07-12 15:10:27,950 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/recovered.edits] 2023-07-12 15:10:27,950 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/recovered.edits] 2023-07-12 15:10:27,956 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/5dd46ccb27744bf7aaaf110ef93b4918 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf1/5dd46ccb27744bf7aaaf110ef93b4918 2023-07-12 15:10:27,956 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/0c3365a685f140dca478de964254bf78 2023-07-12 15:10:27,956 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0e400b85cba04355a8415817330b41ce to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf1/0e400b85cba04355a8415817330b41ce 2023-07-12 15:10:27,958 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/2fd845502eae44cdaa17880e0824e016 2023-07-12 15:10:27,959 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/584e2d7815f14b7e8dfb9d2a94674bbc to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/cf2/584e2d7815f14b7e8dfb9d2a94674bbc 2023-07-12 15:10:27,959 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/dfa43a90c84648ae84c199cff33c8c94 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/cf2/dfa43a90c84648ae84c199cff33c8c94 2023-07-12 15:10:27,960 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/55e01062d8ae4e47b3a843c3f5bbcaa8 2023-07-12 15:10:27,961 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/6bf32312b4bf4007b16c683506684f87 2023-07-12 15:10:27,962 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/91ec52ad4c054c998791e93664eeb112 2023-07-12 15:10:27,962 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/recovered.edits/113.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f/recovered.edits/113.seqid 2023-07-12 15:10:27,962 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/recovered.edits/112.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89/recovered.edits/112.seqid 2023-07-12 15:10:27,963 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ad57113446f90ac0969654c5c8db488f 2023-07-12 15:10:27,963 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/32a3e04fd3a4ef4f387a0d2b230ceb89 2023-07-12 15:10:27,963 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/a3ca3c9f281a4c58a7048844a1719553 2023-07-12 15:10:27,964 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ac1a931fcfbe48ed94ec508b61174f6f 2023-07-12 15:10:27,965 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/ad7844441a75468b9bc66a823260946e 2023-07-12 15:10:27,966 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/b0be540f8ede4a8694967eb4cb92b65d 2023-07-12 15:10:27,966 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf1/c5f5b35c10c14c6d9ad774ae22ef2ca7 2023-07-12 15:10:27,968 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/1d1c839bce084c53947369a7fff22358 2023-07-12 15:10:27,970 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/46403158bcca4a4c9b19920114df55ad 2023-07-12 15:10:27,971 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/497a427964fc498cbe7ba82677e69930 2023-07-12 15:10:27,972 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/835de3cbb41f456aafe8cf60cc22530b 2023-07-12 15:10:27,973 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/8cf49ac0d99544779bec91d11cd2893c 2023-07-12 15:10:27,974 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a5941ebd787b401d9fb5df2a5c7132bf 2023-07-12 15:10:27,975 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/a93bf06bb81a4b8ea7b4057765284531 2023-07-12 15:10:27,977 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/afafef5de87d434bac62fe80fc948a2c 2023-07-12 15:10:27,978 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/bfb78404374449b598819e7c35457979 2023-07-12 15:10:27,979 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/cf2/d9bd15a6ad354590bf9c6b742a08d890 2023-07-12 15:10:27,981 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/recovered.edits/106.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23/recovered.edits/106.seqid 2023-07-12 15:10:27,982 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ba4d910ecaafdf5b7e02268c8df9ad23 2023-07-12 15:10:27,982 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-12 15:10:27,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,985 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-07-12 15:10:27,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-07-12 15:10:27,988 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,988 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-07-12 15:10:27,988 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174627988"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:27,988 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174627988"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:27,988 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174627988"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:27,990 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:10:27,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ba4d910ecaafdf5b7e02268c8df9ad23, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689174623834.ba4d910ecaafdf5b7e02268c8df9ad23.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 32a3e04fd3a4ef4f387a0d2b230ceb89, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1689174626459.32a3e04fd3a4ef4f387a0d2b230ceb89.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => ad57113446f90ac0969654c5c8db488f, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1689174626459.ad57113446f90ac0969654c5c8db488f.', STARTKEY => '60', ENDKEY => ''}] 2023-07-12 15:10:27,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-07-12 15:10:27,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174627990"}]},"ts":"9223372036854775807"} 2023-07-12 15:10:27,991 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-07-12 15:10:27,993 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:10:27,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 59 msec 2023-07-12 15:10:28,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-12 15:10:28,042 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-07-12 15:10:28,055 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 410) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:42178 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:46530 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:53806 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:53840 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:60622 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:53982 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:49968 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:46760 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:49984 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:49980 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:46688 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:49952 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:46732 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:46460 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:44174 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x32c12c41-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-66590417_17 at /127.0.0.1:53836 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=869 (was 867) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=301 (was 327), ProcessCount=168 (was 168), AvailableMemoryMB=5350 (was 5413) 2023-07-12 15:10:28,068 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=301, ProcessCount=168, AvailableMemoryMB=5349 2023-07-12 15:10:28,068 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:10:28,071 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:10:28,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:10:28,072 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:10:28,074 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:10:28,075 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:10:28,076 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:10:28,077 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 empty. 2023-07-12 15:10:28,077 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:10:28,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-12 15:10:28,086 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:28,087 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 428f4f188d352157900381ba01961587, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:10:28,097 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:28,097 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 428f4f188d352157900381ba01961587, disabling compactions & flushes 2023-07-12 15:10:28,097 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,097 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,097 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. after waiting 0 ms 2023-07-12 15:10:28,097 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,097 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,097 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 428f4f188d352157900381ba01961587: 2023-07-12 15:10:28,099 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:10:28,099 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174628099"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174628099"}]},"ts":"1689174628099"} 2023-07-12 15:10:28,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:10:28,101 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:10:28,101 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174628101"}]},"ts":"1689174628101"} 2023-07-12 15:10:28,102 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-07-12 15:10:28,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:28,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:28,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:28,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:28,106 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:10:28,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, ASSIGN}] 2023-07-12 15:10:28,108 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, ASSIGN 2023-07-12 15:10:28,109 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,33703,1689174599345; forceNewPlan=false, retain=false 2023-07-12 15:10:28,259 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:28,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:28,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174628263"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174628263"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174628263"}]},"ts":"1689174628263"} 2023-07-12 15:10:28,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:28,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 428f4f188d352157900381ba01961587, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:28,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:28,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:28,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,431 INFO [StoreOpener-428f4f188d352157900381ba01961587-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,434 DEBUG [StoreOpener-428f4f188d352157900381ba01961587-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1 2023-07-12 15:10:28,435 DEBUG [StoreOpener-428f4f188d352157900381ba01961587-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1 2023-07-12 15:10:28,435 INFO [StoreOpener-428f4f188d352157900381ba01961587-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 428f4f188d352157900381ba01961587 columnFamilyName cf1 2023-07-12 15:10:28,436 INFO [StoreOpener-428f4f188d352157900381ba01961587-1] regionserver.HStore(310): Store=428f4f188d352157900381ba01961587/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:28,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:10:28,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:10:28,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:10:28,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 428f4f188d352157900381ba01961587; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62785341, jitterRate=-0.0644255131483078}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:10:28,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 428f4f188d352157900381ba01961587: 2023-07-12 15:10:28,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587., pid=87, masterSystemTime=1689174628420 2023-07-12 15:10:28,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,450 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:28,450 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174628449"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174628449"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174628449"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174628449"}]},"ts":"1689174628449"} 2023-07-12 15:10:28,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-07-12 15:10:28,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345 in 184 msec 2023-07-12 15:10:28,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-07-12 15:10:28,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, ASSIGN in 347 msec 2023-07-12 15:10:28,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:10:28,456 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174628456"}]},"ts":"1689174628456"} 2023-07-12 15:10:28,457 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-07-12 15:10:28,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:10:28,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 388 msec 2023-07-12 15:10:28,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$12(2454): Client=jenkins//148.251.75.209 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-12 15:10:28,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:10:28,608 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-12 15:10:28,608 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-12 15:10:28,608 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-12 15:10:28,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-07-12 15:10:28,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, REOPEN/MOVE}] 2023-07-12 15:10:28,623 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, REOPEN/MOVE 2023-07-12 15:10:28,624 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:28,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174628624"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174628624"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174628624"}]},"ts":"1689174628624"} 2023-07-12 15:10:28,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:28,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 428f4f188d352157900381ba01961587, disabling compactions & flushes 2023-07-12 15:10:28,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. after waiting 0 ms 2023-07-12 15:10:28,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-12 15:10:28,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:10:28,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:28,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 428f4f188d352157900381ba01961587: 2023-07-12 15:10:28,787 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 428f4f188d352157900381ba01961587 to self. 2023-07-12 15:10:28,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 428f4f188d352157900381ba01961587 2023-07-12 15:10:28,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=CLOSED 2023-07-12 15:10:28,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174628790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174628790"}]},"ts":"1689174628790"} 2023-07-12 15:10:28,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-07-12 15:10:28,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345 in 166 msec 2023-07-12 15:10:28,793 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase20.apache.org,33703,1689174599345; forceNewPlan=false, retain=true 2023-07-12 15:10:28,944 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:10:28,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:28,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174628944"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174628944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174628944"}]},"ts":"1689174628944"} 2023-07-12 15:10:28,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:10:29,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:29,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 428f4f188d352157900381ba01961587, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:10:29,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:29,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 428f4f188d352157900381ba01961587 2023-07-12 15:10:29,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:29,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 428f4f188d352157900381ba01961587 2023-07-12 15:10:29,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 428f4f188d352157900381ba01961587 2023-07-12 15:10:29,113 INFO [StoreOpener-428f4f188d352157900381ba01961587-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 428f4f188d352157900381ba01961587 2023-07-12 15:10:29,114 DEBUG [StoreOpener-428f4f188d352157900381ba01961587-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1 2023-07-12 15:10:29,114 DEBUG [StoreOpener-428f4f188d352157900381ba01961587-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1 2023-07-12 15:10:29,115 INFO [StoreOpener-428f4f188d352157900381ba01961587-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 428f4f188d352157900381ba01961587 columnFamilyName cf1 2023-07-12 15:10:29,115 INFO [StoreOpener-428f4f188d352157900381ba01961587-1] regionserver.HStore(310): Store=428f4f188d352157900381ba01961587/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:10:29,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:10:29,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:10:29,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 428f4f188d352157900381ba01961587 2023-07-12 15:10:29,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 428f4f188d352157900381ba01961587; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63910498, jitterRate=-0.04765936732292175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:10:29,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 428f4f188d352157900381ba01961587: 2023-07-12 15:10:29,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587., pid=92, masterSystemTime=1689174629101 2023-07-12 15:10:29,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:29,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:10:29,127 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:10:29,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174629127"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174629127"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174629127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174629127"}]},"ts":"1689174629127"} 2023-07-12 15:10:29,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-07-12 15:10:29,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345 in 182 msec 2023-07-12 15:10:29,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-07-12 15:10:29,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, REOPEN/MOVE in 510 msec 2023-07-12 15:10:29,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-07-12 15:10:29,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 525 msec 2023-07-12 15:10:29,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, ASSIGN}] 2023-07-12 15:10:29,143 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, ASSIGN 2023-07-12 15:10:29,143 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-12 15:10:29,294 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:10:29,297 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:10:29,297 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:10:29,297 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:10:29,297 DEBUG [jenkins-hbase20:37639] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-07-12 15:10:29,301 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=916b9cfc68d61d2339ab35e622b10301, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:10:29,301 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174629301"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1689174629301"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1689174629301"}]},"ts":"1689174629301"} 2023-07-12 15:10:29,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 916b9cfc68d61d2339ab35e622b10301, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:10:29,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:10:29,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 916b9cfc68d61d2339ab35e622b10301, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-07-12 15:10:29,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:10:29,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 916b9cfc68d61d2339ab35e622b10301 2023-07-12 15:10:29,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:10:29,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 916b9cfc68d61d2339ab35e622b10301 2023-07-12 15:10:29,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 916b9cfc68d61d2339ab35e622b10301 2023-07-12 15:10:29,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:29,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:29,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:29,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:29,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:29,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,603 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-07-12 15:10:30,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:30,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:31,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,187 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:10:32,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:32,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:33,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:34,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:35,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:36,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:37,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:38,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:39,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:40,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:41,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:42,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:43,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:44,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:45,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:46,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:47,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:48,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:49,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:50,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:51,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:52,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:53,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:54,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:55,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:56,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:57,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:58,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:10:59,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:00,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:01,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:02,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:03,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:04,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:05,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:06,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:07,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:08,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:09,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:10,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:11,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:12,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:13,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:14,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:15,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:16,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:17,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:18,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:19,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:20,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:21,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:22,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:23,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:24,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:25,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:26,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:27,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 15:11:28,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 15:11:28,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:28,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 15:11:29,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 15:11:29,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-12 15:11:29,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:29,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,826 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:30,828 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:30,830 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:30,830 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:30,830 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:30,834 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 428f4f188d352157900381ba01961587, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:30,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 exec-time=11 msec 2023-07-12 15:11:30,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:30,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-12 15:11:31,082 INFO [StoreOpener-916b9cfc68d61d2339ab35e622b10301-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 916b9cfc68d61d2339ab35e622b10301 2023-07-12 15:11:31,089 DEBUG [StoreOpener-916b9cfc68d61d2339ab35e622b10301-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1 2023-07-12 15:11:31,089 DEBUG [StoreOpener-916b9cfc68d61d2339ab35e622b10301-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1 2023-07-12 15:11:31,090 INFO [StoreOpener-916b9cfc68d61d2339ab35e622b10301-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 916b9cfc68d61d2339ab35e622b10301 columnFamilyName cf1 2023-07-12 15:11:31,091 INFO [StoreOpener-916b9cfc68d61d2339ab35e622b10301-1] regionserver.HStore(310): Store=916b9cfc68d61d2339ab35e622b10301/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:31,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 916b9cfc68d61d2339ab35e622b10301; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73471530, jitterRate=0.09481111168861389}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-12 15:11:31,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 916b9cfc68d61d2339ab35e622b10301: 2023-07-12 15:11:31,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301., pid=94, masterSystemTime=1689174629456 2023-07-12 15:11:31,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:11:31,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:11:31,097 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=916b9cfc68d61d2339ab35e622b10301, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:31,097 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174691097"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1689174691097"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1689174691097"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1689174691097"}]},"ts":"1689174691097"} 2023-07-12 15:11:31,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-07-12 15:11:31,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 916b9cfc68d61d2339ab35e622b10301, server=jenkins-hbase20.apache.org,41741,1689174599302 in 1 mins, 1.797 sec 2023-07-12 15:11:31,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-07-12 15:11:31,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, ASSIGN in 1 mins, 1.96 sec 2023-07-12 15:11:31,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.511 sec 2023-07-12 15:11:31,106 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1689174628588","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62517,"client":"148.251.75.209:46216","queuetimems":0,"class":"HMaster"} 2023-07-12 15:11:31,106 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-07-12 15:11:33,080 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up 2023-07-12 15:11:33,083 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:33,087 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-12 15:11:33,091 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174693091"}]},"ts":"1689174693091"} 2023-07-12 15:11:33,093 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-07-12 15:11:33,094 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-07-12 15:11:33,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, UNASSIGN}] 2023-07-12 15:11:33,096 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, UNASSIGN 2023-07-12 15:11:33,096 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=916b9cfc68d61d2339ab35e622b10301, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:33,097 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174693096"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1689174693096"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1689174693096"}]},"ts":"1689174693096"} 2023-07-12 15:11:33,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 916b9cfc68d61d2339ab35e622b10301, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:33,101 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, UNASSIGN 2023-07-12 15:11:33,101 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:33,101 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174693101"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174693101"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174693101"}]},"ts":"1689174693101"} 2023-07-12 15:11:33,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345}] 2023-07-12 15:11:33,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-12 15:11:33,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 916b9cfc68d61d2339ab35e622b10301 2023-07-12 15:11:33,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 916b9cfc68d61d2339ab35e622b10301, disabling compactions & flushes 2023-07-12 15:11:33,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:11:33,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:11:33,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. after waiting 0 ms 2023-07-12 15:11:33,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:11:33,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:33,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301. 2023-07-12 15:11:33,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 428f4f188d352157900381ba01961587 2023-07-12 15:11:33,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 916b9cfc68d61d2339ab35e622b10301: 2023-07-12 15:11:33,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 428f4f188d352157900381ba01961587, disabling compactions & flushes 2023-07-12 15:11:33,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:11:33,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:11:33,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. after waiting 0 ms 2023-07-12 15:11:33,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:11:33,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 916b9cfc68d61d2339ab35e622b10301 2023-07-12 15:11:33,264 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=916b9cfc68d61d2339ab35e622b10301, regionState=CLOSED 2023-07-12 15:11:33,264 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174693263"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1689174693263"}]},"ts":"1689174693263"} 2023-07-12 15:11:33,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-12 15:11:33,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:33,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587. 2023-07-12 15:11:33,268 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 428f4f188d352157900381ba01961587: 2023-07-12 15:11:33,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-07-12 15:11:33,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 916b9cfc68d61d2339ab35e622b10301, server=jenkins-hbase20.apache.org,41741,1689174599302 in 167 msec 2023-07-12 15:11:33,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 428f4f188d352157900381ba01961587 2023-07-12 15:11:33,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301, UNASSIGN in 174 msec 2023-07-12 15:11:33,271 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=428f4f188d352157900381ba01961587, regionState=CLOSED 2023-07-12 15:11:33,271 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1689174693271"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174693271"}]},"ts":"1689174693271"} 2023-07-12 15:11:33,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-07-12 15:11:33,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 428f4f188d352157900381ba01961587, server=jenkins-hbase20.apache.org,33703,1689174599345 in 169 msec 2023-07-12 15:11:33,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-07-12 15:11:33,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=428f4f188d352157900381ba01961587, UNASSIGN in 179 msec 2023-07-12 15:11:33,276 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174693276"}]},"ts":"1689174693276"} 2023-07-12 15:11:33,277 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-07-12 15:11:33,278 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-07-12 15:11:33,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 191 msec 2023-07-12 15:11:33,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-12 15:11:33,395 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-07-12 15:11:33,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,398 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-12 15:11:33,402 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:11:33,404 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/recovered.edits] 2023-07-12 15:11:33,408 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/recovered.edits/7.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587/recovered.edits/7.seqid 2023-07-12 15:11:33,408 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/428f4f188d352157900381ba01961587 2023-07-12 15:11:33,408 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-12 15:11:33,410 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,412 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-07-12 15:11:33,413 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-07-12 15:11:33,414 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,414 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-07-12 15:11:33,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174693414"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:33,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174693414"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:33,416 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-07-12 15:11:33,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 428f4f188d352157900381ba01961587, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689174628071.428f4f188d352157900381ba01961587.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 916b9cfc68d61d2339ab35e622b10301, NAME => 'testRollbackForSplitTableRegionWithReplica,,1689174628071_0001.916b9cfc68d61d2339ab35e622b10301.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-07-12 15:11:33,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-07-12 15:11:33,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174693416"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:33,417 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-07-12 15:11:33,419 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:33,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 24 msec 2023-07-12 15:11:33,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-12 15:11:33,500 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-07-12 15:11:33,510 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 414), OpenFileDescriptor=664 (was 869), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=106 (was 301), ProcessCount=168 (was 168), AvailableMemoryMB=5353 (was 5349) - AvailableMemoryMB LEAK? - 2023-07-12 15:11:33,522 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=664, MaxFileDescriptor=60000, SystemLoadAverage=106, ProcessCount=168, AvailableMemoryMB=5353 2023-07-12 15:11:33,522 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:33,525 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:11:33,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:11:33,526 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:33,528 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:11:33,528 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:11:33,530 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,530 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 empty. 2023-07-12 15:11:33,531 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-12 15:11:33,540 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-07-12 15:11:33,541 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5fe85121b37048aebeb5fe93fea04f61, NAME => 'testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:11:33,549 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:33,549 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 5fe85121b37048aebeb5fe93fea04f61, disabling compactions & flushes 2023-07-12 15:11:33,549 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,549 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,549 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. after waiting 0 ms 2023-07-12 15:11:33,550 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,550 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,550 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:33,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:11:33,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689174693552"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174693552"}]},"ts":"1689174693552"} 2023-07-12 15:11:33,554 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:11:33,554 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:11:33,555 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174693554"}]},"ts":"1689174693554"} 2023-07-12 15:11:33,555 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-07-12 15:11:33,557 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:11:33,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:11:33,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:11:33,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:11:33,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:11:33,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, ASSIGN}] 2023-07-12 15:11:33,560 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, ASSIGN 2023-07-12 15:11:33,560 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:11:33,710 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:11:33,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=5fe85121b37048aebeb5fe93fea04f61, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:33,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689174693712"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174693712"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174693712"}]},"ts":"1689174693712"} 2023-07-12 15:11:33,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 5fe85121b37048aebeb5fe93fea04f61, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:33,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5fe85121b37048aebeb5fe93fea04f61, NAME => 'testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:11:33,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:33,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:33,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,879 INFO [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,881 DEBUG [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1 2023-07-12 15:11:33,881 DEBUG [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1 2023-07-12 15:11:33,882 INFO [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fe85121b37048aebeb5fe93fea04f61 columnFamilyName cf1 2023-07-12 15:11:33,883 INFO [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] regionserver.HStore(310): Store=5fe85121b37048aebeb5fe93fea04f61/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:33,883 INFO [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,885 DEBUG [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2 2023-07-12 15:11:33,885 DEBUG [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2 2023-07-12 15:11:33,886 INFO [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fe85121b37048aebeb5fe93fea04f61 columnFamilyName cf2 2023-07-12 15:11:33,887 INFO [StoreOpener-5fe85121b37048aebeb5fe93fea04f61-1] regionserver.HStore(310): Store=5fe85121b37048aebeb5fe93fea04f61/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:33,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:33,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:33,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:11:33,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 5fe85121b37048aebeb5fe93fea04f61; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69617353, jitterRate=0.03737939894199371}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:33,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:33,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61., pid=104, masterSystemTime=1689174693870 2023-07-12 15:11:33,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:33,898 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=5fe85121b37048aebeb5fe93fea04f61, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:33,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689174693898"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174693898"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174693898"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174693898"}]},"ts":"1689174693898"} 2023-07-12 15:11:33,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-07-12 15:11:33,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 5fe85121b37048aebeb5fe93fea04f61, server=jenkins-hbase20.apache.org,41741,1689174599302 in 183 msec 2023-07-12 15:11:33,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-07-12 15:11:33,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, ASSIGN in 343 msec 2023-07-12 15:11:33,903 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:11:33,903 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174693903"}]},"ts":"1689174693903"} 2023-07-12 15:11:33,904 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-07-12 15:11:33,906 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:11:33,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 380 msec 2023-07-12 15:11:34,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:34,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 15:11:34,038 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 15:11:34,038 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,038 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 15:11:34,038 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 15:11:34,039 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,039 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,039 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,039 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,040 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,039 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,040 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,041 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,041 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. started... 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,041 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,042 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:11:34,042 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,042 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,042 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,043 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,043 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,043 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,043 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,044 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,044 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,050 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/f49c168a702f423882065a5552e4fe28 2023-07-12 15:11:34,062 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/24021d14308a48c09da77f50572464e2 2023-07-12 15:11:34,067 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/f49c168a702f423882065a5552e4fe28 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/f49c168a702f423882065a5552e4fe28 2023-07-12 15:11:34,071 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/f49c168a702f423882065a5552e4fe28, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:34,072 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/24021d14308a48c09da77f50572464e2 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/24021d14308a48c09da77f50572464e2 2023-07-12 15:11:34,076 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/24021d14308a48c09da77f50572464e2, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:34,077 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 35ms, sequenceid=6, compaction requested=false 2023-07-12 15:11:34,077 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-07-12 15:11:34,078 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:34,078 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,078 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:34,078 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:34,078 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,078 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,078 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,079 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,079 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,079 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 15:11:34,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28b969dd[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:34,082 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 15:11:34,082 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,083 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,084 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:34,084 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,084 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:34,084 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,084 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:34,084 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,084 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,084 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:34,085 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,085 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,085 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,086 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,086 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,086 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,086 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,086 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,086 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,086 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:34,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 15:11:34,092 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 15:11:34,092 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:34,092 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:34,092 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 15:11:34,092 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:34,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,094 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,095 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,096 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,096 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,097 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,097 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,098 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,098 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,105 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 15:11:34,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:34,105 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,107 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testInvalidSplitKey because node does not exist (not an error) 2023-07-12 15:11:34,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,107 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,107 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,206 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:34,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:34,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:34,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:34,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 15:11:34,220 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 15:11:34,220 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,220 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 15:11:34,220 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 15:11:34,220 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,220 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,222 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,222 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,223 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,223 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,224 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,224 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,224 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. started... 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,225 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,225 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,226 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,226 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,233 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/9c1c604b93f544deb9133db2801b9db0 2023-07-12 15:11:34,244 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/b36051a349d64ea09b7dfdecf44c5cff 2023-07-12 15:11:34,248 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/9c1c604b93f544deb9133db2801b9db0 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/9c1c604b93f544deb9133db2801b9db0 2023-07-12 15:11:34,252 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/9c1c604b93f544deb9133db2801b9db0, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:34,252 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/b36051a349d64ea09b7dfdecf44c5cff as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/b36051a349d64ea09b7dfdecf44c5cff 2023-07-12 15:11:34,256 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/b36051a349d64ea09b7dfdecf44c5cff, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:34,256 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 31ms, sequenceid=19, compaction requested=false 2023-07-12 15:11:34,256 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:34,256 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,257 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:34,257 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:34,257 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,257 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,257 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,258 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 15:11:34,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5842b1d5[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:34,260 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 15:11:34,260 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,260 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,261 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:34,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:34,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,261 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,262 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,262 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:34,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 15:11:34,265 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 15:11:34,265 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:34,265 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:34,265 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 15:11:34,265 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,266 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,267 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,267 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,268 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,268 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,268 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,270 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 15:11:34,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:34,271 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:34,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,271 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:34,272 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,272 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,272 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,272 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,372 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:34,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:34,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:34,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:34,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 15:11:34,399 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 15:11:34,399 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,399 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 15:11:34,399 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 15:11:34,399 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,400 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,401 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,401 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,401 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,401 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,401 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,401 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,403 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:34,404 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,404 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:34,404 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. started... 2023-07-12 15:11:34,405 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:34,413 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,415 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/51346792187b47e598518d1979701b1d 2023-07-12 15:11:34,419 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,419 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,419 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,419 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,419 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,433 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/0c0898e79200442c937c9d274fb67dfd 2023-07-12 15:11:34,437 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/51346792187b47e598518d1979701b1d as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/51346792187b47e598518d1979701b1d 2023-07-12 15:11:34,441 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/51346792187b47e598518d1979701b1d, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:34,442 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/0c0898e79200442c937c9d274fb67dfd as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/0c0898e79200442c937c9d274fb67dfd 2023-07-12 15:11:34,446 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/0c0898e79200442c937c9d274fb67dfd, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:34,447 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 42ms, sequenceid=32, compaction requested=true 2023-07-12 15:11:34,447 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:34,447 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,447 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:34,447 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:34,447 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,447 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,447 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,449 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,449 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,449 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,449 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 15:11:34,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75d6f6fe[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:34,451 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 15:11:34,451 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,451 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,451 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,451 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,452 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,452 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,452 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:34,453 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,453 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:34,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 15:11:34,456 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 15:11:34,456 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:34,456 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:34,456 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 15:11:34,456 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:34,460 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,460 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,461 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,461 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,461 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,461 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,463 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,463 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,463 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,466 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 15:11:34,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,466 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,467 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,567 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:34,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:34,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:34,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:34,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 15:11:34,580 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 15:11:34,580 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,580 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 15:11:34,580 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 15:11:34,581 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,581 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,582 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,582 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,582 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,582 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,582 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,582 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:34,583 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,583 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. started... 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:34,583 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,584 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,584 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:34,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,584 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,584 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,585 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,585 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,585 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,585 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,585 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,591 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/bf14998f1d0b43de80df735848639767 2023-07-12 15:11:34,602 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/273ed067cd21414290ea948ea247e0be 2023-07-12 15:11:34,605 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/bf14998f1d0b43de80df735848639767 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/bf14998f1d0b43de80df735848639767 2023-07-12 15:11:34,608 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/bf14998f1d0b43de80df735848639767, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:34,608 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/273ed067cd21414290ea948ea247e0be as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/273ed067cd21414290ea948ea247e0be 2023-07-12 15:11:34,611 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/273ed067cd21414290ea948ea247e0be, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:34,612 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 28ms, sequenceid=45, compaction requested=true 2023-07-12 15:11:34,612 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:34,612 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,612 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:34,612 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:34,612 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,612 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,612 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,613 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,613 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,613 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,613 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 15:11:34,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38bc22c8[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:34,615 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 15:11:34,615 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,616 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,616 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:34,616 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,617 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,617 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:34,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 15:11:34,620 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 15:11:34,620 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:34,620 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:34,620 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 15:11:34,620 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,622 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,622 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,624 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,624 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,624 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 15:11:34,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:34,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,626 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,626 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,627 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,726 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:34,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:34,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:34,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:34,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 15:11:34,738 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 15:11:34,738 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,738 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 15:11:34,738 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 15:11:34,739 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,739 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,740 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,740 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,740 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,740 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,740 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,740 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,740 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:34,742 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:34,741 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,742 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,742 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,742 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,742 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,742 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. started... 2023-07-12 15:11:34,742 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,742 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:34,742 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,743 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,743 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,743 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,743 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,743 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,749 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/a7b4a744c26b411a8f0e867b90fc8fb0 2023-07-12 15:11:34,761 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/6600ee14af984fb588850ea1fba0fe44 2023-07-12 15:11:34,765 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/a7b4a744c26b411a8f0e867b90fc8fb0 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/a7b4a744c26b411a8f0e867b90fc8fb0 2023-07-12 15:11:34,769 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/a7b4a744c26b411a8f0e867b90fc8fb0, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:34,769 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/6600ee14af984fb588850ea1fba0fe44 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/6600ee14af984fb588850ea1fba0fe44 2023-07-12 15:11:34,773 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/6600ee14af984fb588850ea1fba0fe44, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:34,774 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 32ms, sequenceid=58, compaction requested=true 2023-07-12 15:11:34,774 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:34,774 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,774 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:34,774 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:34,774 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,774 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,774 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,776 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,776 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,777 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,777 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 15:11:34,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b285383[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:34,778 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 15:11:34,779 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,780 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,780 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:34,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:34,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,780 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,781 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,781 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:34,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 15:11:34,784 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 15:11:34,784 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:34,784 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:34,784 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 15:11:34,784 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,785 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,785 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,785 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,786 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,787 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,787 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,787 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,789 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 15:11:34,789 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:34,789 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:34,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testInvalidSplitKey because node does not exist (not an error) 2023-07-12 15:11:34,789 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:34,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,790 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,890 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:34,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:34,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-12 15:11:34,908 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-12 15:11:34,908 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,908 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-12 15:11:34,908 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-12 15:11:34,909 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,909 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,909 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,909 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,909 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,909 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,910 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,910 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,911 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,911 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-12 15:11:34,911 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,912 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:34,912 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. started... 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,912 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,912 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:34,913 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,913 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,914 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,914 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,914 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,914 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,923 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/e7fc8505c97045b2ac4313f18af099be 2023-07-12 15:11:34,933 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/975b5b5dc2e84e4980108c78fa393fe3 2023-07-12 15:11:34,939 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/e7fc8505c97045b2ac4313f18af099be as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/e7fc8505c97045b2ac4313f18af099be 2023-07-12 15:11:34,943 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/e7fc8505c97045b2ac4313f18af099be, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:34,943 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/975b5b5dc2e84e4980108c78fa393fe3 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/975b5b5dc2e84e4980108c78fa393fe3 2023-07-12 15:11:34,947 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/975b5b5dc2e84e4980108c78fa393fe3, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:34,948 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 36ms, sequenceid=71, compaction requested=true 2023-07-12 15:11:34,948 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:34,948 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:34,948 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:34,948 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:34,948 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,948 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-12 15:11:34,948 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-12 15:11:34,949 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,949 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,949 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,950 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:34,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-12 15:11:34,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16298d2a[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:34,951 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-12 15:11:34,951 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,952 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,952 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:34,952 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:34,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,953 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,954 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-12 15:11:34,954 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,954 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,954 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-12 15:11:34,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:34,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-12 15:11:34,956 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-12 15:11:34,956 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:34,956 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:34,956 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-12 15:11:34,956 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:34,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,957 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,958 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:34,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,958 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,958 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:34,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:34,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:34,960 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-12 15:11:34,960 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,960 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:34,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,962 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-12 15:11:34,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-12 15:11:34,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:34,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:34,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:34,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:34,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,962 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,963 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:34,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:34,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:34,963 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:34,963 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:34,963 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-12 15:11:34,963 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-12 15:11:35,063 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:35,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:35,078 INFO [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:35,079 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/9c1c604b93f544deb9133db2801b9db0 because midkey is the same as first or last row 2023-07-12 15:11:35,079 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/b36051a349d64ea09b7dfdecf44c5cff because midkey is the same as first or last row 2023-07-12 15:11:35,079 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HRegion(2745): Flushing 5fe85121b37048aebeb5fe93fea04f61 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:11:35,089 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/c1cfe7215da94c1fb971e7a07bce8665 2023-07-12 15:11:35,104 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/57b75a8edba14311997b9c288c88a8b0 2023-07-12 15:11:35,109 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf1/c1cfe7215da94c1fb971e7a07bce8665 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/c1cfe7215da94c1fb971e7a07bce8665 2023-07-12 15:11:35,113 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/c1cfe7215da94c1fb971e7a07bce8665, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:35,114 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/.tmp/cf2/57b75a8edba14311997b9c288c88a8b0 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/57b75a8edba14311997b9c288c88a8b0 2023-07-12 15:11:35,118 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/57b75a8edba14311997b9c288c88a8b0, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:35,119 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5fe85121b37048aebeb5fe93fea04f61 in 40ms, sequenceid=82, compaction requested=true 2023-07-12 15:11:35,120 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.HRegion(2446): Flush status journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:35,121 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/9c1c604b93f544deb9133db2801b9db0 because midkey is the same as first or last row 2023-07-12 15:11:35,121 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41741] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/b36051a349d64ea09b7dfdecf44c5cff because midkey is the same as first or last row 2023-07-12 15:11:35,121 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:35,121 DEBUG [Listener at localhost.localdomain/46471] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-07-12 15:11:35,122 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:35,124 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-07-12 15:11:35,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testInvalidSplitKey 2023-07-12 15:11:35,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-12 15:11:35,129 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174695128"}]},"ts":"1689174695128"} 2023-07-12 15:11:35,130 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-07-12 15:11:35,131 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-07-12 15:11:35,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, UNASSIGN}] 2023-07-12 15:11:35,133 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, UNASSIGN 2023-07-12 15:11:35,134 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=5fe85121b37048aebeb5fe93fea04f61, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:35,134 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689174695134"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174695134"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174695134"}]},"ts":"1689174695134"} 2023-07-12 15:11:35,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 5fe85121b37048aebeb5fe93fea04f61, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:35,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-12 15:11:35,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:35,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 5fe85121b37048aebeb5fe93fea04f61, disabling compactions & flushes 2023-07-12 15:11:35,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:35,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:35,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. after waiting 0 ms 2023-07-12 15:11:35,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:35,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:11:35,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:35,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61. 2023-07-12 15:11:35,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 5fe85121b37048aebeb5fe93fea04f61: 2023-07-12 15:11:35,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:35,304 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=5fe85121b37048aebeb5fe93fea04f61, regionState=CLOSED 2023-07-12 15:11:35,304 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1689174695304"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174695304"}]},"ts":"1689174695304"} 2023-07-12 15:11:35,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-07-12 15:11:35,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 5fe85121b37048aebeb5fe93fea04f61, server=jenkins-hbase20.apache.org,41741,1689174599302 in 170 msec 2023-07-12 15:11:35,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-07-12 15:11:35,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=5fe85121b37048aebeb5fe93fea04f61, UNASSIGN in 176 msec 2023-07-12 15:11:35,311 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174695311"}]},"ts":"1689174695311"} 2023-07-12 15:11:35,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-07-12 15:11:35,313 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-07-12 15:11:35,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 188 msec 2023-07-12 15:11:35,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-12 15:11:35,433 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-07-12 15:11:35,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testInvalidSplitKey 2023-07-12 15:11:35,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,442 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-12 15:11:35,448 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:35,451 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/recovered.edits] 2023-07-12 15:11:35,455 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/51346792187b47e598518d1979701b1d to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/51346792187b47e598518d1979701b1d 2023-07-12 15:11:35,457 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/9c1c604b93f544deb9133db2801b9db0 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/9c1c604b93f544deb9133db2801b9db0 2023-07-12 15:11:35,460 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/a7b4a744c26b411a8f0e867b90fc8fb0 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/a7b4a744c26b411a8f0e867b90fc8fb0 2023-07-12 15:11:35,462 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/bf14998f1d0b43de80df735848639767 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/bf14998f1d0b43de80df735848639767 2023-07-12 15:11:35,464 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/c1cfe7215da94c1fb971e7a07bce8665 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/c1cfe7215da94c1fb971e7a07bce8665 2023-07-12 15:11:35,466 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/e7fc8505c97045b2ac4313f18af099be to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/e7fc8505c97045b2ac4313f18af099be 2023-07-12 15:11:35,468 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/f49c168a702f423882065a5552e4fe28 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf1/f49c168a702f423882065a5552e4fe28 2023-07-12 15:11:35,471 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/0c0898e79200442c937c9d274fb67dfd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/0c0898e79200442c937c9d274fb67dfd 2023-07-12 15:11:35,473 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/24021d14308a48c09da77f50572464e2 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/24021d14308a48c09da77f50572464e2 2023-07-12 15:11:35,474 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/273ed067cd21414290ea948ea247e0be to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/273ed067cd21414290ea948ea247e0be 2023-07-12 15:11:35,476 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/57b75a8edba14311997b9c288c88a8b0 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/57b75a8edba14311997b9c288c88a8b0 2023-07-12 15:11:35,478 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/6600ee14af984fb588850ea1fba0fe44 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/6600ee14af984fb588850ea1fba0fe44 2023-07-12 15:11:35,480 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/975b5b5dc2e84e4980108c78fa393fe3 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/975b5b5dc2e84e4980108c78fa393fe3 2023-07-12 15:11:35,482 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/b36051a349d64ea09b7dfdecf44c5cff to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/cf2/b36051a349d64ea09b7dfdecf44c5cff 2023-07-12 15:11:35,486 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/recovered.edits/85.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61/recovered.edits/85.seqid 2023-07-12 15:11:35,486 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testInvalidSplitKey/5fe85121b37048aebeb5fe93fea04f61 2023-07-12 15:11:35,486 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-12 15:11:35,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,491 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-07-12 15:11:35,494 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-07-12 15:11:35,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-07-12 15:11:35,495 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174695495"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:35,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-12 15:11:35,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5fe85121b37048aebeb5fe93fea04f61, NAME => 'testInvalidSplitKey,,1689174693526.5fe85121b37048aebeb5fe93fea04f61.', STARTKEY => '', ENDKEY => ''}] 2023-07-12 15:11:35,497 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-07-12 15:11:35,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174695497"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:35,498 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-07-12 15:11:35,499 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:35,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 63 msec 2023-07-12 15:11:35,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-12 15:11:35,546 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-07-12 15:11:35,567 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=370 (was 349) - Thread LEAK? -, OpenFileDescriptor=748 (was 664) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=106 (was 106), ProcessCount=168 (was 168), AvailableMemoryMB=5349 (was 5353) 2023-07-12 15:11:35,579 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=370, OpenFileDescriptor=748, MaxFileDescriptor=60000, SystemLoadAverage=106, ProcessCount=168, AvailableMemoryMB=5348 2023-07-12 15:11:35,580 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:35,582 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:11:35,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:11:35,584 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:35,585 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:11:35,586 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:11:35,587 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,588 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd empty. 2023-07-12 15:11:35,588 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-12 15:11:35,599 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-12 15:11:35,600 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => e1edf3f9a2728ef8bba8c663379c31fd, NAME => 'testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:11:35,610 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:35,610 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing e1edf3f9a2728ef8bba8c663379c31fd, disabling compactions & flushes 2023-07-12 15:11:35,610 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,610 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,610 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. after waiting 0 ms 2023-07-12 15:11:35,610 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,610 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,610 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:35,613 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:11:35,615 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174695614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174695614"}]},"ts":"1689174695614"} 2023-07-12 15:11:35,616 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:11:35,617 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:11:35,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174695617"}]},"ts":"1689174695617"} 2023-07-12 15:11:35,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-12 15:11:35,622 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:11:35,624 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:11:35,624 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:11:35,624 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:11:35,624 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:11:35,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, ASSIGN}] 2023-07-12 15:11:35,627 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, ASSIGN 2023-07-12 15:11:35,628 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:11:35,778 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:11:35,780 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=e1edf3f9a2728ef8bba8c663379c31fd, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:35,781 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174695780"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174695780"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174695780"}]},"ts":"1689174695780"} 2023-07-12 15:11:35,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:35,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e1edf3f9a2728ef8bba8c663379c31fd, NAME => 'testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:11:35,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:35,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:35,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,950 INFO [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,953 DEBUG [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1 2023-07-12 15:11:35,953 DEBUG [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1 2023-07-12 15:11:35,954 INFO [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1edf3f9a2728ef8bba8c663379c31fd columnFamilyName cf1 2023-07-12 15:11:35,954 INFO [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] regionserver.HStore(310): Store=e1edf3f9a2728ef8bba8c663379c31fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:35,955 INFO [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,957 DEBUG [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2 2023-07-12 15:11:35,957 DEBUG [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2 2023-07-12 15:11:35,957 INFO [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1edf3f9a2728ef8bba8c663379c31fd columnFamilyName cf2 2023-07-12 15:11:35,958 INFO [StoreOpener-e1edf3f9a2728ef8bba8c663379c31fd-1] regionserver.HStore(310): Store=e1edf3f9a2728ef8bba8c663379c31fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:35,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:35,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:35,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:11:35,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened e1edf3f9a2728ef8bba8c663379c31fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69219769, jitterRate=0.03145493566989899}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:35,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:35,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd., pid=111, masterSystemTime=1689174695937 2023-07-12 15:11:35,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:35,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=e1edf3f9a2728ef8bba8c663379c31fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:35,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174695973"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174695973"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174695973"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174695973"}]},"ts":"1689174695973"} 2023-07-12 15:11:35,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-07-12 15:11:35,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 in 190 msec 2023-07-12 15:11:35,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-07-12 15:11:35,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, ASSIGN in 351 msec 2023-07-12 15:11:35,979 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:11:35,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174695979"}]},"ts":"1689174695979"} 2023-07-12 15:11:35,981 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-12 15:11:35,983 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:11:35,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 400 msec 2023-07-12 15:11:36,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:36,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 15:11:36,100 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,100 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,100 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 15:11:36,100 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:11:36,101 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,101 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,102 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,102 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,102 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,102 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:36,104 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,104 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:36,104 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. started... 2023-07-12 15:11:36,106 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:11:36,113 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,114 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/998a6dcf23da4db497e7ab4d4b4457de 2023-07-12 15:11:36,119 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,119 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,119 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,119 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,119 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,125 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/e658e183bbcd4fa08f27dc068cbffc55 2023-07-12 15:11:36,130 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/998a6dcf23da4db497e7ab4d4b4457de as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de 2023-07-12 15:11:36,134 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:36,135 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/e658e183bbcd4fa08f27dc068cbffc55 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 2023-07-12 15:11:36,140 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:36,141 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 35ms, sequenceid=6, compaction requested=false 2023-07-12 15:11:36,141 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,142 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:36,142 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,142 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:36,142 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:36,142 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,142 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,142 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,144 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,144 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,144 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,144 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 15:11:36,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@540af817[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:36,148 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:11:36,148 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,148 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,148 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,148 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,149 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,149 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:36,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:36,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:36,149 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,150 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,150 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:36,150 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,150 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,150 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,151 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,151 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,151 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,151 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,151 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,151 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,151 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:36,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:11:36,155 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 15:11:36,155 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:36,155 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:36,155 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,155 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:36,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,163 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,165 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,165 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,168 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,168 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,168 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,172 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,173 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:36,173 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,173 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,174 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,274 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:36,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:36,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:36,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:36,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 15:11:36,297 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,297 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,297 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 15:11:36,297 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:11:36,298 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,298 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,299 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,299 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,299 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,299 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,299 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,299 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,300 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:36,301 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:36,301 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,301 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. started... 2023-07-12 15:11:36,301 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:36,303 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,303 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,303 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,303 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,303 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,303 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,309 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/c2563d66758f41aca2b373bc3a12555c 2023-07-12 15:11:36,322 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/2131f7f31f3341c6b67cef036901c67f 2023-07-12 15:11:36,326 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/c2563d66758f41aca2b373bc3a12555c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c 2023-07-12 15:11:36,329 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:36,330 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/2131f7f31f3341c6b67cef036901c67f as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f 2023-07-12 15:11:36,333 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:36,334 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 33ms, sequenceid=19, compaction requested=false 2023-07-12 15:11:36,334 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:36,334 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,334 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:36,334 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:36,334 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,334 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,334 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,336 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,336 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,336 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,336 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 15:11:36,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d792a3c[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:36,337 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:11:36,337 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,338 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,338 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:36,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,338 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,338 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,338 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,339 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,339 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:36,339 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,338 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:36,339 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,340 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,340 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:36,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:11:36,342 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 15:11:36,342 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:36,342 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:36,342 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,343 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,344 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,345 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,345 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,347 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,347 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,347 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,363 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,363 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,363 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,363 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,364 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:36,363 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,363 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,364 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:36,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,374 ERROR [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid d6c01eab-2c69-419b-b4a6-953858ab157f) service to localhost.localdomain/127.0.0.1:34607 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:11:36,465 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:36,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:36,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:36,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:36,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 15:11:36,489 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,489 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,489 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 15:11:36,489 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:11:36,490 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,490 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,491 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,491 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,491 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,491 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,491 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,491 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,492 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,492 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:36,493 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,493 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. started... 2023-07-12 15:11:36,493 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:36,493 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,494 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,494 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,494 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,494 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,494 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,500 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/2dccd5ba9fde4fbaa94cad305e9b983c 2023-07-12 15:11:36,512 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/7feaab11dc294e78b2440f0ca34ee26b 2023-07-12 15:11:36,517 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/2dccd5ba9fde4fbaa94cad305e9b983c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c 2023-07-12 15:11:36,523 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:36,524 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/7feaab11dc294e78b2440f0ca34ee26b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b 2023-07-12 15:11:36,527 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:36,528 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 35ms, sequenceid=32, compaction requested=true 2023-07-12 15:11:36,528 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:36,528 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,528 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:36,528 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:36,528 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,528 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,528 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,529 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,529 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,529 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,529 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 15:11:36,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14d4d45b[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:36,531 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:11:36,531 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,532 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,532 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,532 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:36,533 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,533 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:36,533 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,533 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,533 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,534 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,534 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,534 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,534 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,534 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,534 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,534 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:36,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:11:36,537 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 15:11:36,537 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:36,537 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:36,537 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,537 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:36,538 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,538 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,538 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,538 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,538 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,538 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,539 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,540 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,541 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,541 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,543 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:36,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,543 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,544 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:36,544 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,544 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,544 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,544 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,544 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,644 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:36,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:36,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:36,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:36,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 15:11:36,663 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,663 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,663 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 15:11:36,663 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:11:36,664 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,664 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,665 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,665 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,665 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,665 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,665 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,665 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:36,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,667 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,667 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,667 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,667 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,667 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,667 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:36,668 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,668 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. started... 2023-07-12 15:11:36,668 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:36,669 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:36,669 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,670 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,670 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,670 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,670 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,670 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,682 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/c1eebe1a57774d448a7032bc584be288 2023-07-12 15:11:36,695 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/e5177bd9ed4141938648bb3b35275bb1 2023-07-12 15:11:36,700 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/c1eebe1a57774d448a7032bc584be288 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 2023-07-12 15:11:36,707 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:36,708 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/e5177bd9ed4141938648bb3b35275bb1 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 2023-07-12 15:11:36,713 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:36,713 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 45ms, sequenceid=45, compaction requested=true 2023-07-12 15:11:36,713 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:36,713 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,714 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:36,714 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:36,714 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,714 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,714 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,715 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,715 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,715 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,715 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 15:11:36,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e4e4b36[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:36,717 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:11:36,717 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,718 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,718 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:36,718 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,719 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,719 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:36,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:11:36,722 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 15:11:36,722 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:36,722 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:36,722 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,722 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:36,723 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,723 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,723 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,724 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,724 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,725 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,725 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,725 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,726 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,727 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,727 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,730 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,729 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:36,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:36,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,731 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:36,730 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,731 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,731 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,831 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:36,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:36,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:36,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:36,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 15:11:36,851 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,851 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,851 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 15:11:36,851 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:11:36,851 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,851 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,852 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,852 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,853 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,853 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:36,854 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:36,854 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. started... 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,854 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,854 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:36,855 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,862 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/a79298dbeb774f739b9b860d975f7812 2023-07-12 15:11:36,874 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/267ec1e12bfb403e9e65f686a0abca25 2023-07-12 15:11:36,878 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/a79298dbeb774f739b9b860d975f7812 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 2023-07-12 15:11:36,883 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:36,883 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/267ec1e12bfb403e9e65f686a0abca25 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 2023-07-12 15:11:36,887 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:36,888 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 34ms, sequenceid=58, compaction requested=true 2023-07-12 15:11:36,888 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:36,888 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:36,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:36,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:36,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:36,888 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:36,889 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,889 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,889 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,889 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:36,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 15:11:36,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d07d8a8[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:36,891 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:11:36,891 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,892 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:36,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:36,892 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,893 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,893 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:36,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:11:36,895 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 15:11:36,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:36,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:36,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 15:11:36,895 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,896 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,897 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,897 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,897 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:36,899 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,899 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,899 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:36,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:36,901 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:36,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:36,901 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,902 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:36,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:36,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,002 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:37,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:37,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:37,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:37,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-12 15:11:37,028 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:37,028 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:37,028 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-12 15:11:37,028 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-12 15:11:37,028 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,028 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,029 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,029 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:37,029 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:37,029 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:37,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:37,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:37,029 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,030 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:37,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-12 15:11:37,031 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:37,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:37,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:37,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:37,032 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:37,033 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:37,033 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. started... 2023-07-12 15:11:37,033 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:37,033 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,033 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,033 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,034 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:37,034 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,034 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:37,043 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb 2023-07-12 15:11:37,060 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/4f4291a146404597aedb3e823bdeeb15 2023-07-12 15:11:37,066 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb 2023-07-12 15:11:37,072 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:37,073 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/4f4291a146404597aedb3e823bdeeb15 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 2023-07-12 15:11:37,079 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:37,080 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 47ms, sequenceid=71, compaction requested=true 2023-07-12 15:11:37,080 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:37,080 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:37,080 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:37,080 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:37,080 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,080 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-12 15:11:37,080 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-12 15:11:37,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,082 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:37,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,082 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:37,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-12 15:11:37,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9eb33fe[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:37,086 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-12 15:11:37,086 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,088 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,088 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,088 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,089 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,089 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:37,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:37,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-12 15:11:37,089 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:37,090 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:37,090 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,090 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:37,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:37,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:37,090 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:37,091 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:37,091 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:37,091 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:37,092 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-12 15:11:37,092 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:37,092 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-12 15:11:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:37,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:37,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:37,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:37,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:37,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:37,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-12 15:11:37,098 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-12 15:11:37,098 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:37,098 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:37,098 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-12 15:11:37,098 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:37,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:37,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:37,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,106 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,108 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:37,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,108 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,110 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,110 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:37,112 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-12 15:11:37,112 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,112 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,113 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:37,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:37,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:37,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-12 15:11:37,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:37,114 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:37,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:37,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:37,114 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:37,115 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,115 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-12 15:11:37,215 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:37,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:37,482 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-12 15:11:37,484 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:37,487 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,491 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:37,491 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:37,492 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:37,737 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,737 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:37,737 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:37,738 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:37,738 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-07-12 15:11:37,738 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:37,738 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:37,738 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:37,738 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:37,739 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:11:37,741 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-12 15:11:37,741 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:37,742 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:37,742 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:37,742 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:37,743 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:37,743 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:37,743 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:37,744 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:37,744 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:37,745 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:37,745 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:37,745 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:37,746 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:37,746 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:37,747 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:37,747 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:37,747 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:37,748 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:37,748 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:37,748 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:37,749 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:37,749 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:37,749 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:37,750 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:37,750 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:37,750 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:37,750 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:37,751 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,752 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:37,752 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,752 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,752 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,752 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-07-12 15:11:37,753 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:37,753 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:37,753 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174727753 2023-07-12 15:11:37,755 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,755 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:37,757 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:37,757 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:37,757 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:37,757 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174757757 2023-07-12 15:11:37,757 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174817757 2023-07-12 15:11:37,757 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:11:37,757 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:37,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:37,759 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:37,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:37,760 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:38,008 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,008 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:38,008 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:38,008 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:38,009 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:38,009 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:38,010 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:38,010 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:38,010 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:38,012 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:38,015 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-12 15:11:38,015 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:38,016 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:38,016 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:38,016 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:38,016 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,016 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:38,017 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,017 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,017 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,017 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:38,017 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,017 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,018 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,018 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:38,018 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:38,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:38,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,019 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,020 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,020 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:38,020 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,020 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,020 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:38,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,021 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,022 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,022 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,022 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:38,022 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,022 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:38,022 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,022 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,023 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,023 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 15:11:38,024 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:38,024 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:38,025 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174728025 2023-07-12 15:11:38,025 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:38,026 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,028 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:38,028 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:38,028 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:38,028 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174758028 2023-07-12 15:11:38,028 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174818028 2023-07-12 15:11:38,028 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:11:38,029 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:38,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN}] 2023-07-12 15:11:38,031 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:38,031 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:38,031 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:38,279 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,279 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:38,279 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:38,279 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:38,280 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:38,280 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:38,280 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:38,280 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:38,280 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:38,280 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:11:38,281 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-12 15:11:38,282 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:38,282 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:38,282 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:38,283 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:38,283 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,283 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:38,283 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,283 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:38,284 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,285 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,286 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,286 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,286 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:38,286 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,286 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,287 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,287 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,287 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,287 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:38,287 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,287 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:38,287 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,287 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,287 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,287 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-12 15:11:38,288 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:38,289 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:38,289 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174728289 2023-07-12 15:11:38,290 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:38,290 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,293 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:38,293 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:38,293 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:38,293 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174758293 2023-07-12 15:11:38,293 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174818293 2023-07-12 15:11:38,293 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:11:38,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN}] 2023-07-12 15:11:38,294 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:38,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:38,298 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:38,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174698297"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174698297"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174698297"}]},"ts":"1689174698297"} 2023-07-12 15:11:38,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:38,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing e1edf3f9a2728ef8bba8c663379c31fd, disabling compactions & flushes 2023-07-12 15:11:38,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:38,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:38,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. after waiting 0 ms 2023-07-12 15:11:38,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:38,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing e1edf3f9a2728ef8bba8c663379c31fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:11:38,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead 2023-07-12 15:11:38,479 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/e9569bdcad814d679be9a71e2fcf4204 2023-07-12 15:11:38,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead 2023-07-12 15:11:38,486 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:38,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/.tmp/cf2/e9569bdcad814d679be9a71e2fcf4204 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 2023-07-12 15:11:38,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:38,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e1edf3f9a2728ef8bba8c663379c31fd in 40ms, sequenceid=82, compaction requested=true 2023-07-12 15:11:38,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:11:38,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:38,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. 2023-07-12 15:11:38,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for e1edf3f9a2728ef8bba8c663379c31fd: 2023-07-12 15:11:38,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,497 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED 2023-07-12 15:11:38,497 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174698497"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174698497"}]},"ts":"1689174698497"} 2023-07-12 15:11:38,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-07-12 15:11:38,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 in 199 msec 2023-07-12 15:11:38,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-07-12 15:11:38,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN in 205 msec 2023-07-12 15:11:38,503 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:38,503 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:38,503 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:38,544 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,545 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:38,545 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:38,545 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:38,545 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:38,545 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:38,546 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:38,546 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:38,546 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:38,547 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:38,550 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-12 15:11:38,551 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:38,551 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:38,551 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:38,552 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:38,552 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:38,552 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:38,553 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,553 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:38,554 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,554 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:38,555 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,555 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:38,555 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,555 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:38,555 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,556 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:38,556 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:38,556 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,556 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:38,557 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,557 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:38,557 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,557 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:38,557 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,558 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:38,558 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,558 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:38,558 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,558 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,559 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:38,559 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,559 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:38,559 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,559 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:38,560 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,560 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,560 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,560 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:38,561 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:38,561 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-07-12 15:11:38,561 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:38,562 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:38,562 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174728562 2023-07-12 15:11:38,562 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:38,563 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:38,565 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:38,565 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:38,565 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:38,565 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174758565 2023-07-12 15:11:38,565 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174818565 2023-07-12 15:11:38,566 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:11:38,568 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:38,577 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=e1edf3f9a2728ef8bba8c663379c31fd, threads=14 2023-07-12 15:11:38,577 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,577 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,577 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,577 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,578 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,578 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,581 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,581 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,581 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,581 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,582 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,586 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,587 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,587 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,668 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,688 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,688 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,694 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,694 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,701 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,708 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,709 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,724 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,724 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:38,802 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:11:39,082 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,084 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,084 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,381 ERROR [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid 631c6a2a-c60e-45ea-8ae8-e13efaf3ce20) service to localhost.localdomain/127.0.0.1:34607 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:11:39,485 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,485 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region e1edf3f9a2728ef8bba8c663379c31fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:11:39,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:39,552 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:39,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:39,566 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,566 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:39,567 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:39,567 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:39,567 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:39,567 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:39,568 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:39,568 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:39,568 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:39,569 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:39,572 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-12 15:11:39,573 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:39,573 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:39,573 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:39,574 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:39,574 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:39,574 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:39,575 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:39,575 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:39,575 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:39,575 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:39,575 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:39,575 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:39,576 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:39,577 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:39,578 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:39,578 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:39,578 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:39,578 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:39,578 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:39,578 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:39,579 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:39,580 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,580 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,580 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,580 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:39,580 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:39,580 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 15:11:39,580 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:39,581 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:39,581 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174729581 2023-07-12 15:11:39,582 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:39,583 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:39,585 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:39,585 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:39,585 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:39,586 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174759586 2023-07-12 15:11:39,586 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174819586 2023-07-12 15:11:39,586 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-12 15:11:39,593 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=e1edf3f9a2728ef8bba8c663379c31fd, threads=14 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,593 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,594 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,656 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,658 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,660 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,660 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,666 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,667 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,667 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,667 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:39,668 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:40,057 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:40,058 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:40,058 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:40,064 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:40,068 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f for region: e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:40,068 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region e1edf3f9a2728ef8bba8c663379c31fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:11:40,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:40,540 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:11:40,543 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:11:40,543 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:40,543 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:40,543 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:40,586 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,587 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:40,587 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:40,587 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:40,587 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:40,588 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:40,588 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:40,588 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:40,588 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:40,590 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:40,593 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-12 15:11:40,593 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:40,593 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:40,594 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:40,595 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:40,595 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:40,596 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:40,596 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:40,597 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:40,598 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:40,598 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:40,599 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:40,599 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:40,600 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:40,600 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:40,601 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:40,602 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:40,602 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:40,602 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:40,603 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:40,603 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:40,603 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:40,603 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:40,603 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:40,604 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:40,604 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:40,604 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:40,604 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:40,604 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,605 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,605 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:40,605 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,605 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,605 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:40,605 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:40,606 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:40,606 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:40,606 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,606 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:40,606 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:40,606 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:40,607 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,607 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,607 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,607 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:40,607 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:40,607 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 17 msec 2023-07-12 15:11:40,608 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:40,608 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:40,609 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174730609 2023-07-12 15:11:40,610 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:40,610 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,613 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:40,613 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:40,613 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:40,613 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174760613 2023-07-12 15:11:40,613 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174820613 2023-07-12 15:11:40,613 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:11:40,616 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:40,618 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:40,618 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:40,618 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:40,864 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,864 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:40,864 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:40,864 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:40,865 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:40,865 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:40,865 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:40,866 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:40,866 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:40,868 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:40,871 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-12 15:11:40,872 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:40,872 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:40,872 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:40,873 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:40,874 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:40,874 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:40,874 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:40,875 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:40,876 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:40,877 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:40,878 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,878 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,878 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:40,878 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:40,879 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-12 15:11:40,879 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:40,880 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:40,880 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174730880 2023-07-12 15:11:40,881 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,882 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:40,884 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:40,884 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:40,884 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:40,884 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174760884 2023-07-12 15:11:40,884 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174820884 2023-07-12 15:11:40,884 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:11:40,884 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:40,887 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174700887"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1689174700887"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1689174700887"}]},"ts":"1689174700887"} 2023-07-12 15:11:40,887 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174700887"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174700887"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174700887"}]},"ts":"1689174700887"} 2023-07-12 15:11:40,887 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174700887"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174700887"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174700887"}]},"ts":"1689174700887"} 2023-07-12 15:11:40,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:40,889 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:40,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:40,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:41,134 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,135 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:41,135 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:41,135 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:41,138 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:41,138 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:41,139 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:41,139 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:41,139 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:41,140 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:41,142 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-12 15:11:41,142 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:41,142 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:41,143 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:41,144 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:41,144 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:41,144 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:41,144 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,145 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:41,145 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,145 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,145 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,146 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:41,146 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,146 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,146 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,147 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:41,147 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:41,148 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,148 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,148 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,149 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:41,149 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,149 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,149 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,150 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:41,150 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,150 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,150 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,151 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:41,152 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,152 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,152 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:41,152 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:41,152 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:11:41,153 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:41,153 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:41,154 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174731154 2023-07-12 15:11:41,155 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:41,155 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,157 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:41,157 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:41,157 INFO [Listener at localhost.localdomain/46471] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377, table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:41,158 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=5fece8a4ef8de483896514a3f6819f7a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 15:11:41,158 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 15:11:41,158 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:41,158 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174761158 2023-07-12 15:11:41,158 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174821158 2023-07-12 15:11:41,158 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:11:41,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689174701158"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1689174701158"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1689174701158"}]},"ts":"1689174701158"} 2023-07-12 15:11:41,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174701158"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174701158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174701158"}]},"ts":"1689174701158"} 2023-07-12 15:11:41,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174701158"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174701158"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174701158"}]},"ts":"1689174701158"} 2023-07-12 15:11:41,161 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:41,162 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:41,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:41,162 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:41,408 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,409 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:41,409 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:41,409 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:41,411 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:41,412 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:41,412 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:41,412 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:41,412 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:41,414 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-12 15:11:41,419 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-07-12 15:11:41,419 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:41,420 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:41,420 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:41,422 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:41,422 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:41,422 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:41,423 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,423 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:41,423 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,423 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,423 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,423 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,424 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:41,425 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:41,426 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:41,426 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,427 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,427 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,427 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:41,427 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:41,427 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-07-12 15:11:41,427 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:41,428 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:41,428 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174731428 2023-07-12 15:11:41,429 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,429 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:41,431 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:41,431 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:41,431 INFO [Listener at localhost.localdomain/46471] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377, table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:41,432 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=5fece8a4ef8de483896514a3f6819f7a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 15:11:41,432 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 15:11:41,432 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:41,432 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174761432 2023-07-12 15:11:41,432 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174821432 2023-07-12 15:11:41,432 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-12 15:11:41,432 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:41,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN}] 2023-07-12 15:11:41,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:41,434 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:41,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:41,682 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,682 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:41,682 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:41,682 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:41,683 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:41,683 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:41,683 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:41,683 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:41,683 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:41,684 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:11:41,686 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-07-12 15:11:41,687 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:41,687 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:41,687 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:41,688 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:41,688 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:41,688 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:41,688 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,689 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:41,689 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,689 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:41,689 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,690 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:41,690 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,690 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:41,690 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,691 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:41,691 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:41,691 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,691 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:41,692 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,692 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:41,692 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,692 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:41,693 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,693 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:41,693 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,693 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:41,694 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,694 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,694 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:41,694 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:41,695 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:41,695 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,695 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,696 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,696 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:41,696 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:41,696 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-12 15:11:41,697 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:41,697 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:41,697 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174731697 2023-07-12 15:11:41,697 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:41,698 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:41,701 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:41,701 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:41,701 INFO [Listener at localhost.localdomain/46471] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377, table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:41,701 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=5fece8a4ef8de483896514a3f6819f7a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 15:11:41,701 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-12 15:11:41,701 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:41,701 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174761701 2023-07-12 15:11:41,701 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174821701 2023-07-12 15:11:41,701 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:11:41,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN}] 2023-07-12 15:11:41,702 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:41,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN 2023-07-12 15:11:41,705 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:11:41,705 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN 2023-07-12 15:11:41,706 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:11:41,855 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:11:41,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=5fece8a4ef8de483896514a3f6819f7a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:41,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174701859"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174701859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174701859"}]},"ts":"1689174701859"} 2023-07-12 15:11:41,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 5fece8a4ef8de483896514a3f6819f7a, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:41,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:41,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174701867"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174701867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174701867"}]},"ts":"1689174701867"} 2023-07-12 15:11:41,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 1e5bcd3cd0db1bdec735d838c68a83ca, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:42,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1e5bcd3cd0db1bdec735d838c68a83ca, NAME => 'testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 15:11:42,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:42,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:42,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,027 INFO [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,028 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1 2023-07-12 15:11:42,028 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1 2023-07-12 15:11:42,029 INFO [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e5bcd3cd0db1bdec735d838c68a83ca columnFamilyName cf1 2023-07-12 15:11:42,043 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead-top 2023-07-12 15:11:42,048 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c-top 2023-07-12 15:11:42,052 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de-top 2023-07-12 15:11:42,057 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb-top 2023-07-12 15:11:42,061 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812-top 2023-07-12 15:11:42,065 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288-top 2023-07-12 15:11:42,069 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c-top 2023-07-12 15:11:42,070 INFO [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(310): Store=1e5bcd3cd0db1bdec735d838c68a83ca/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:42,070 INFO [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,071 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2 2023-07-12 15:11:42,071 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2 2023-07-12 15:11:42,071 INFO [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e5bcd3cd0db1bdec735d838c68a83ca columnFamilyName cf2 2023-07-12 15:11:42,087 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f-top 2023-07-12 15:11:42,092 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25-top 2023-07-12 15:11:42,096 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15-top 2023-07-12 15:11:42,099 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b-top 2023-07-12 15:11:42,103 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1-top 2023-07-12 15:11:42,106 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55-top 2023-07-12 15:11:42,110 DEBUG [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204-top 2023-07-12 15:11:42,110 INFO [StoreOpener-1e5bcd3cd0db1bdec735d838c68a83ca-1] regionserver.HStore(310): Store=1e5bcd3cd0db1bdec735d838c68a83ca/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:42,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:42,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1e5bcd3cd0db1bdec735d838c68a83ca; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58741721, jitterRate=-0.12468014657497406}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:42,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1e5bcd3cd0db1bdec735d838c68a83ca: 2023-07-12 15:11:42,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca., pid=118, masterSystemTime=1689174702015 2023-07-12 15:11:42,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:42,116 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:42,116 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:42,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:42,116 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,117 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): 1e5bcd3cd0db1bdec735d838c68a83ca/cf2 is initiating minor compaction (all files) 2023-07-12 15:11:42,116 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,117 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): 1e5bcd3cd0db1bdec735d838c68a83ca/cf1 is initiating minor compaction (all files) 2023-07-12 15:11:42,117 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1e5bcd3cd0db1bdec735d838c68a83ca/cf2 in testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,117 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1e5bcd3cd0db1bdec735d838c68a83ca/cf1 in testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,117 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204-top] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/.tmp, totalSize=34.9 K 2023-07-12 15:11:42,117 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead-top] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/.tmp, totalSize=34.9 K 2023-07-12 15:11:42,117 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174696090 2023-07-12 15:11:42,117 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174696090 2023-07-12 15:11:42,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,118 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174696278 2023-07-12 15:11:42,118 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174696278 2023-07-12 15:11:42,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:42,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,118 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174696468 2023-07-12 15:11:42,118 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,118 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174696468 2023-07-12 15:11:42,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5fece8a4ef8de483896514a3f6819f7a, NAME => 'testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 15:11:42,118 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174702118"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174702118"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174702118"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174702118"}]},"ts":"1689174702118"} 2023-07-12 15:11:42,118 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174696646 2023-07-12 15:11:42,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:42,118 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174696646 2023-07-12 15:11:42,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,119 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689174696833 2023-07-12 15:11:42,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:42,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,119 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689174697006 2023-07-12 15:11:42,119 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689174696833 2023-07-12 15:11:42,119 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174697218 2023-07-12 15:11:42,119 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689174697006 2023-07-12 15:11:42,120 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174697218 2023-07-12 15:11:42,120 INFO [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,121 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1 2023-07-12 15:11:42,121 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1 2023-07-12 15:11:42,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-07-12 15:11:42,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 1e5bcd3cd0db1bdec735d838c68a83ca, server=jenkins-hbase20.apache.org,37993,1689174599377 in 251 msec 2023-07-12 15:11:42,121 INFO [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fece8a4ef8de483896514a3f6819f7a columnFamilyName cf1 2023-07-12 15:11:42,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN in 420 msec 2023-07-12 15:11:42,128 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 1e5bcd3cd0db1bdec735d838c68a83ca#cf2#compaction#107 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:42,129 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1e5bcd3cd0db1bdec735d838c68a83ca#cf1#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:42,133 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead-bottom 2023-07-12 15:11:42,141 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c-bottom 2023-07-12 15:11:42,144 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de-bottom 2023-07-12 15:11:42,144 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/.tmp/cf2/f7f676d2619340759fee4245a999f97b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/f7f676d2619340759fee4245a999f97b 2023-07-12 15:11:42,144 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/.tmp/cf1/dcf9ba2854214c2ba2d0efba7a1ca7c6 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/dcf9ba2854214c2ba2d0efba7a1ca7c6 2023-07-12 15:11:42,148 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb-bottom 2023-07-12 15:11:42,150 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1e5bcd3cd0db1bdec735d838c68a83ca/cf1 of 1e5bcd3cd0db1bdec735d838c68a83ca into dcf9ba2854214c2ba2d0efba7a1ca7c6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:42,150 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1e5bcd3cd0db1bdec735d838c68a83ca: 2023-07-12 15:11:42,150 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca., storeName=1e5bcd3cd0db1bdec735d838c68a83ca/cf1, priority=9, startTime=1689174702116; duration=0sec 2023-07-12 15:11:42,150 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:42,151 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812-bottom 2023-07-12 15:11:42,159 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1e5bcd3cd0db1bdec735d838c68a83ca/cf2 of 1e5bcd3cd0db1bdec735d838c68a83ca into f7f676d2619340759fee4245a999f97b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:42,159 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1e5bcd3cd0db1bdec735d838c68a83ca: 2023-07-12 15:11:42,159 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca., storeName=1e5bcd3cd0db1bdec735d838c68a83ca/cf2, priority=9, startTime=1689174702116; duration=0sec 2023-07-12 15:11:42,159 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:42,159 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288-bottom 2023-07-12 15:11:42,163 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c-bottom 2023-07-12 15:11:42,163 INFO [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(310): Store=5fece8a4ef8de483896514a3f6819f7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:42,163 INFO [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,164 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2 2023-07-12 15:11:42,164 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2 2023-07-12 15:11:42,164 INFO [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fece8a4ef8de483896514a3f6819f7a columnFamilyName cf2 2023-07-12 15:11:42,175 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f-bottom 2023-07-12 15:11:42,178 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25-bottom 2023-07-12 15:11:42,181 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15-bottom 2023-07-12 15:11:42,185 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b-bottom 2023-07-12 15:11:42,188 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1-bottom 2023-07-12 15:11:42,192 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55-bottom 2023-07-12 15:11:42,196 DEBUG [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204-bottom 2023-07-12 15:11:42,196 INFO [StoreOpener-5fece8a4ef8de483896514a3f6819f7a-1] regionserver.HStore(310): Store=5fece8a4ef8de483896514a3f6819f7a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:42,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:42,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:42,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 5fece8a4ef8de483896514a3f6819f7a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66483657, jitterRate=-0.009316310286521912}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:42,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 5fece8a4ef8de483896514a3f6819f7a: 2023-07-12 15:11:42,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a., pid=117, masterSystemTime=1689174702015 2023-07-12 15:11:42,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:11:42,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:11:42,203 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:42,203 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:42,204 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,204 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,204 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): 5fece8a4ef8de483896514a3f6819f7a/cf1 is initiating minor compaction (all files) 2023-07-12 15:11:42,204 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): 5fece8a4ef8de483896514a3f6819f7a/cf2 is initiating minor compaction (all files) 2023-07-12 15:11:42,204 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5fece8a4ef8de483896514a3f6819f7a/cf1 in testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,204 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 5fece8a4ef8de483896514a3f6819f7a/cf2 in testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,204 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/.tmp, totalSize=34.9 K 2023-07-12 15:11:42,204 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/.tmp, totalSize=34.9 K 2023-07-12 15:11:42,205 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174696090 2023-07-12 15:11:42,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:42,205 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174696090 2023-07-12 15:11:42,205 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174696278 2023-07-12 15:11:42,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=5fece8a4ef8de483896514a3f6819f7a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,206 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174696278 2023-07-12 15:11:42,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174702205"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174702205"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174702205"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174702205"}]},"ts":"1689174702205"} 2023-07-12 15:11:42,206 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174696468 2023-07-12 15:11:42,206 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174696468 2023-07-12 15:11:42,206 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174696646 2023-07-12 15:11:42,207 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174696646 2023-07-12 15:11:42,207 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174696833 2023-07-12 15:11:42,207 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174696833 2023-07-12 15:11:42,208 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174697006 2023-07-12 15:11:42,208 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting 4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174697006 2023-07-12 15:11:42,208 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting 0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174697218 2023-07-12 15:11:42,209 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174697218 2023-07-12 15:11:42,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-07-12 15:11:42,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5fece8a4ef8de483896514a3f6819f7a, server=jenkins-hbase20.apache.org,37993,1689174599377 in 345 msec 2023-07-12 15:11:42,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-07-12 15:11:42,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN in 509 msec 2023-07-12 15:11:42,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-12 15:11:42,212 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:42,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-12 15:11:42,218 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 5fece8a4ef8de483896514a3f6819f7a#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:42,218 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5fece8a4ef8de483896514a3f6819f7a#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:42,232 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/.tmp/cf1/03fd0c13a3e541d0b9ad03cd6a480309 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/03fd0c13a3e541d0b9ad03cd6a480309 2023-07-12 15:11:42,232 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/.tmp/cf2/72e06f86da9347c384a4439c8ef993b7 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/72e06f86da9347c384a4439c8ef993b7 2023-07-12 15:11:42,236 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5fece8a4ef8de483896514a3f6819f7a/cf2 of 5fece8a4ef8de483896514a3f6819f7a into 72e06f86da9347c384a4439c8ef993b7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:42,236 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5fece8a4ef8de483896514a3f6819f7a/cf1 of 5fece8a4ef8de483896514a3f6819f7a into 03fd0c13a3e541d0b9ad03cd6a480309(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:42,236 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5fece8a4ef8de483896514a3f6819f7a: 2023-07-12 15:11:42,236 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5fece8a4ef8de483896514a3f6819f7a: 2023-07-12 15:11:42,236 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a., storeName=5fece8a4ef8de483896514a3f6819f7a/cf2, priority=9, startTime=1689174702202; duration=0sec 2023-07-12 15:11:42,236 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a., storeName=5fece8a4ef8de483896514a3f6819f7a/cf1, priority=9, startTime=1689174702202; duration=0sec 2023-07-12 15:11:42,237 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:42,237 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:42,378 ERROR [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid 5606ee4f-c045-409e-a742-091e025e583c) service to localhost.localdomain/127.0.0.1:34607 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:42,452 ERROR [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-12 15:11:42,452 INFO [Listener at localhost.localdomain/46471] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-12 15:11:42,453 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-12 15:11:42,454 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-07-12 15:11:42,454 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-12 15:11:42,455 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=729ceb1cf3f9ae336ecb337304ed6137, daughterA=70a49d37d17b69af191be811441bc737, daughterB=e3d4e0ad69916b0568e043baa8382198 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=93ab9e169b318d49d9c50d2917ea323f, daughterA=eebb53209a67cd5fe14ed980ba3856a2, daughterB=52c16816c786a7989acf47df11e6e54a 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:42,456 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=1eef801734bac1a9fe215d8599908306, daughterA=8ed48bfdabff1ec7a9bd902580a8ca94, daughterB=8f66539c980ad08fef74cce889cb8e16 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=1eef801734bac1a9fe215d8599908306, ASSIGN 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=002a47310ffb17757afc3abd905a1bc6, daughterA=db5251afa0b685e4e695d9600d5bcda1, daughterB=420b5727eefe787777cba173bc4334d4 2023-07-12 15:11:42,457 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ba4d910ecaafdf5b7e02268c8df9ad23, daughterA=32a3e04fd3a4ef4f387a0d2b230ceb89, daughterB=ad57113446f90ac0969654c5c8db488f 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,41741,1689174599302, table=testRollbackForSplitTableRegionWithReplica, region=916b9cfc68d61d2339ab35e622b10301 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=428f4f188d352157900381ba01961587, daughterA=040a454da8d8c7aaab5d57dac3cfaade, daughterB=6f6c59597a7020052910a335d52416b1 2023-07-12 15:11:42,458 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:42,459 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-12 15:11:42,459 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:42,459 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:42,459 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-12 15:11:42,459 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5fece8a4ef8de483896514a3f6819f7a, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 1e5bcd3cd0db1bdec735d838c68a83ca, server=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca held the lock before restarting, call acquireLock to restore it. 2023-07-12 15:11:42,460 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,460 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,460 INFO [Listener at localhost.localdomain/46471] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,460 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:42,461 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e1edf3f9a2728ef8bba8c663379c31fd, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:42,461 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:42,461 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:42,461 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 5fece8a4ef8de483896514a3f6819f7a, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:42,461 DEBUG [Listener at localhost.localdomain/46471] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 1e5bcd3cd0db1bdec735d838c68a83ca, server=jenkins-hbase20.apache.org,37993,1689174599377 didn't hold the lock before restarting, skip acquiring lock. 2023-07-12 15:11:42,461 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-12 15:11:42,462 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-12 15:11:42,462 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,33703,1689174599345, table=hbase:meta, region=1588230740 2023-07-12 15:11:42,462 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689174732462 2023-07-12 15:11:42,464 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:42,464 DEBUG [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1517): Joining cluster... 2023-07-12 15:11:42,467 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=688006d0ea2ee3fc16a6d36bae9436b7, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:42,467 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=e1edf3f9a2728ef8bba8c663379c31fd, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=2 2023-07-12 15:11:42,467 INFO [Listener at localhost.localdomain/46471] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37993,1689174599377, table=testRecoveryAndDoubleExecution, region=e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:42,467 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=5fece8a4ef8de483896514a3f6819f7a, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=86 2023-07-12 15:11:42,467 DEBUG [Listener at localhost.localdomain/46471] assignment.RegionStateStore(147): Load hbase:meta entry region=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37993,1689174599377, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377, openSeqNum=86 2023-07-12 15:11:42,468 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-12 15:11:42,468 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689174762468 2023-07-12 15:11:42,468 INFO [Listener at localhost.localdomain/46471] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689174822468 2023-07-12 15:11:42,468 INFO [Listener at localhost.localdomain/46471] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-12 15:11:42,468 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-12 15:11:42,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e1edf3f9a2728ef8bba8c663379c31fd, daughterA=5fece8a4ef8de483896514a3f6819f7a, daughterB=1e5bcd3cd0db1bdec735d838c68a83ca in 4.9830 sec 2023-07-12 15:11:42,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37993] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 148.251.75.209:40686 deadline: 1689174762719, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd. is not online on jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,833 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:11:42,843 INFO [Listener at localhost.localdomain/46471] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 15:11:42,845 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:11:42,856 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:42,857 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-07-12 15:11:42,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRecoveryAndDoubleExecution 2023-07-12 15:11:42,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:42,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-12 15:11:42,861 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174702861"}]},"ts":"1689174702861"} 2023-07-12 15:11:42,861 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-12 15:11:42,862 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-07-12 15:11:42,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, UNASSIGN}] 2023-07-12 15:11:42,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, UNASSIGN 2023-07-12 15:11:42,864 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,864 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174702864"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174702864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174702864"}]},"ts":"1689174702864"} 2023-07-12 15:11:42,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 1e5bcd3cd0db1bdec735d838c68a83ca, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:42,867 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, UNASSIGN 2023-07-12 15:11:42,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=5fece8a4ef8de483896514a3f6819f7a, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:42,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174702867"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174702867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174702867"}]},"ts":"1689174702867"} 2023-07-12 15:11:42,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 5fece8a4ef8de483896514a3f6819f7a, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:42,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-12 15:11:43,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:43,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1e5bcd3cd0db1bdec735d838c68a83ca, disabling compactions & flushes 2023-07-12 15:11:43,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:43,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:43,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. after waiting 0 ms 2023-07-12 15:11:43,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:43,020 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead-top] to archive 2023-07-12 15:11:43,023 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:43,027 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,030 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,033 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,035 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,037 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,039 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,042 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,045 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204-top] to archive 2023-07-12 15:11:43,045 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:43,048 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,050 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,052 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,054 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,055 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,056 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,057 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:11:43,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:43,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca. 2023-07-12 15:11:43,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1e5bcd3cd0db1bdec735d838c68a83ca: 2023-07-12 15:11:43,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:43,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:43,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 5fece8a4ef8de483896514a3f6819f7a, disabling compactions & flushes 2023-07-12 15:11:43,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:43,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:43,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. after waiting 0 ms 2023-07-12 15:11:43,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:43,064 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=1e5bcd3cd0db1bdec735d838c68a83ca, regionState=CLOSED 2023-07-12 15:11:43,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174703063"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174703063"}]},"ts":"1689174703063"} 2023-07-12 15:11:43,064 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead-bottom] to archive 2023-07-12 15:11:43,065 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:43,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-07-12 15:11:43,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 1e5bcd3cd0db1bdec735d838c68a83ca, server=jenkins-hbase20.apache.org,37993,1689174599377 in 200 msec 2023-07-12 15:11:43,066 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1e5bcd3cd0db1bdec735d838c68a83ca, UNASSIGN in 203 msec 2023-07-12 15:11:43,067 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,068 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,069 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,070 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,071 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,072 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,074 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204-bottom] to archive 2023-07-12 15:11:43,075 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:43,076 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,077 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,078 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,078 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,079 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,080 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,081 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:11:43,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:43,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a. 2023-07-12 15:11:43,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 5fece8a4ef8de483896514a3f6819f7a: 2023-07-12 15:11:43,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:43,087 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=5fece8a4ef8de483896514a3f6819f7a, regionState=CLOSED 2023-07-12 15:11:43,087 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1689174703087"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174703087"}]},"ts":"1689174703087"} 2023-07-12 15:11:43,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-07-12 15:11:43,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 5fece8a4ef8de483896514a3f6819f7a, server=jenkins-hbase20.apache.org,37993,1689174599377 in 220 msec 2023-07-12 15:11:43,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-07-12 15:11:43,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fece8a4ef8de483896514a3f6819f7a, UNASSIGN in 226 msec 2023-07-12 15:11:43,090 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174703090"}]},"ts":"1689174703090"} 2023-07-12 15:11:43,091 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-12 15:11:43,092 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-07-12 15:11:43,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 234 msec 2023-07-12 15:11:43,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-12 15:11:43,165 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-07-12 15:11:43,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRecoveryAndDoubleExecution 2023-07-12 15:11:43,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:43,170 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:43,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:43,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-12 15:11:43,181 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,181 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:43,181 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:43,184 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/recovered.edits] 2023-07-12 15:11:43,185 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/recovered.edits] 2023-07-12 15:11:43,185 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/recovered.edits] 2023-07-12 15:11:43,190 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/.regioninfo to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/.regioninfo 2023-07-12 15:11:43,190 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/.regioninfo to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/.regioninfo 2023-07-12 15:11:43,190 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead 2023-07-12 15:11:43,191 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/2dccd5ba9fde4fbaa94cad305e9b983c 2023-07-12 15:11:43,193 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,193 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/998a6dcf23da4db497e7ab4d4b4457de 2023-07-12 15:11:43,193 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/0b7c6bb97b4e49f0b577ecf088ba9ead.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,193 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,194 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb 2023-07-12 15:11:43,194 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/2dccd5ba9fde4fbaa94cad305e9b983c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,194 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,195 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/a79298dbeb774f739b9b860d975f7812 2023-07-12 15:11:43,195 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/998a6dcf23da4db497e7ab4d4b4457de.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,195 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,196 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c1eebe1a57774d448a7032bc584be288 2023-07-12 15:11:43,196 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a3ebcdd38c8c4e0285e22e99a10dcdcb.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,197 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,197 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf1/c2563d66758f41aca2b373bc3a12555c 2023-07-12 15:11:43,197 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/a79298dbeb774f739b9b860d975f7812.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,198 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,198 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c1eebe1a57774d448a7032bc584be288.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,199 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/2131f7f31f3341c6b67cef036901c67f 2023-07-12 15:11:43,199 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,200 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/c2563d66758f41aca2b373bc3a12555c.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,200 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/267ec1e12bfb403e9e65f686a0abca25 2023-07-12 15:11:43,202 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/4f4291a146404597aedb3e823bdeeb15 2023-07-12 15:11:43,202 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,202 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/2131f7f31f3341c6b67cef036901c67f.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,203 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/7feaab11dc294e78b2440f0ca34ee26b 2023-07-12 15:11:43,204 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,204 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/267ec1e12bfb403e9e65f686a0abca25.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,204 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e5177bd9ed4141938648bb3b35275bb1 2023-07-12 15:11:43,205 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,205 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/4f4291a146404597aedb3e823bdeeb15.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,206 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e658e183bbcd4fa08f27dc068cbffc55 2023-07-12 15:11:43,206 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,207 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/7feaab11dc294e78b2440f0ca34ee26b.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,207 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/cf2/e9569bdcad814d679be9a71e2fcf4204 2023-07-12 15:11:43,208 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,208 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e5177bd9ed4141938648bb3b35275bb1.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,208 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,209 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,211 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/e658e183bbcd4fa08f27dc068cbffc55.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,212 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/dcf9ba2854214c2ba2d0efba7a1ca7c6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf1/dcf9ba2854214c2ba2d0efba7a1ca7c6 2023-07-12 15:11:43,213 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/recovered.edits/85.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd/recovered.edits/85.seqid 2023-07-12 15:11:43,213 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/5fece8a4ef8de483896514a3f6819f7a/cf2/e9569bdcad814d679be9a71e2fcf4204.e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,213 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/e1edf3f9a2728ef8bba8c663379c31fd 2023-07-12 15:11:43,215 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/f7f676d2619340759fee4245a999f97b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/cf2/f7f676d2619340759fee4245a999f97b 2023-07-12 15:11:43,215 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/03fd0c13a3e541d0b9ad03cd6a480309 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf1/03fd0c13a3e541d0b9ad03cd6a480309 2023-07-12 15:11:43,217 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/72e06f86da9347c384a4439c8ef993b7 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/cf2/72e06f86da9347c384a4439c8ef993b7 2023-07-12 15:11:43,217 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca/recovered.edits/90.seqid 2023-07-12 15:11:43,217 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/1e5bcd3cd0db1bdec735d838c68a83ca 2023-07-12 15:11:43,219 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a/recovered.edits/90.seqid 2023-07-12 15:11:43,219 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testRecoveryAndDoubleExecution/5fece8a4ef8de483896514a3f6819f7a 2023-07-12 15:11:43,219 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-12 15:11:43,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:43,222 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-07-12 15:11:43,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-07-12 15:11:43,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:43,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-07-12 15:11:43,224 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174703224"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:43,225 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174703224"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:43,225 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174703224"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:43,226 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:11:43,226 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e1edf3f9a2728ef8bba8c663379c31fd, NAME => 'testRecoveryAndDoubleExecution,,1689174695583.e1edf3f9a2728ef8bba8c663379c31fd.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 5fece8a4ef8de483896514a3f6819f7a, NAME => 'testRecoveryAndDoubleExecution,,1689174697485.5fece8a4ef8de483896514a3f6819f7a.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 1e5bcd3cd0db1bdec735d838c68a83ca, NAME => 'testRecoveryAndDoubleExecution,41,1689174697485.1e5bcd3cd0db1bdec735d838c68a83ca.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 15:11:43,226 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-07-12 15:11:43,226 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174703226"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:43,227 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-07-12 15:11:43,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-12 15:11:43,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 62 msec 2023-07-12 15:11:43,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-12 15:11:43,274 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-07-12 15:11:43,298 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=377 (was 370) - Thread LEAK? -, OpenFileDescriptor=844 (was 748) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=98 (was 106), ProcessCount=168 (was 168), AvailableMemoryMB=5310 (was 5348) 2023-07-12 15:11:43,309 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=98, ProcessCount=168, AvailableMemoryMB=5309 2023-07-12 15:11:43,309 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:43,311 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:11:43,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:11:43,313 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,314 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:11:43,315 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:11:43,316 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,317 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 empty. 2023-07-12 15:11:43,317 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,317 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-12 15:11:43,329 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-07-12 15:11:43,331 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => efc743968937366eb65d282b3ec929b1, NAME => 'testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:11:43,341 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:43,342 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing efc743968937366eb65d282b3ec929b1, disabling compactions & flushes 2023-07-12 15:11:43,342 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,342 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,342 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. after waiting 0 ms 2023-07-12 15:11:43,342 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,342 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,342 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:43,345 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:11:43,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174703346"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174703346"}]},"ts":"1689174703346"} 2023-07-12 15:11:43,348 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:11:43,349 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:11:43,349 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174703349"}]},"ts":"1689174703349"} 2023-07-12 15:11:43,351 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-07-12 15:11:43,354 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:11:43,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:11:43,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:11:43,355 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:11:43,356 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:11:43,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, ASSIGN}] 2023-07-12 15:11:43,357 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, ASSIGN 2023-07-12 15:11:43,358 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:11:43,508 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:11:43,511 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=efc743968937366eb65d282b3ec929b1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,511 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174703510"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174703510"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174703510"}]},"ts":"1689174703510"} 2023-07-12 15:11:43,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure efc743968937366eb65d282b3ec929b1, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:43,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => efc743968937366eb65d282b3ec929b1, NAME => 'testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:11:43,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:43,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:43,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,678 INFO [StoreOpener-efc743968937366eb65d282b3ec929b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,680 DEBUG [StoreOpener-efc743968937366eb65d282b3ec929b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1 2023-07-12 15:11:43,680 DEBUG [StoreOpener-efc743968937366eb65d282b3ec929b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1 2023-07-12 15:11:43,680 INFO [StoreOpener-efc743968937366eb65d282b3ec929b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region efc743968937366eb65d282b3ec929b1 columnFamilyName cf1 2023-07-12 15:11:43,681 INFO [StoreOpener-efc743968937366eb65d282b3ec929b1-1] regionserver.HStore(310): Store=efc743968937366eb65d282b3ec929b1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:43,681 INFO [StoreOpener-efc743968937366eb65d282b3ec929b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,682 DEBUG [StoreOpener-efc743968937366eb65d282b3ec929b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2 2023-07-12 15:11:43,682 DEBUG [StoreOpener-efc743968937366eb65d282b3ec929b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2 2023-07-12 15:11:43,683 INFO [StoreOpener-efc743968937366eb65d282b3ec929b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region efc743968937366eb65d282b3ec929b1 columnFamilyName cf2 2023-07-12 15:11:43,683 INFO [StoreOpener-efc743968937366eb65d282b3ec929b1-1] regionserver.HStore(310): Store=efc743968937366eb65d282b3ec929b1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:43,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:43,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:43,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:11:43,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened efc743968937366eb65d282b3ec929b1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73860673, jitterRate=0.10060979425907135}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:43,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:43,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1., pid=127, masterSystemTime=1689174703668 2023-07-12 15:11:43,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=efc743968937366eb65d282b3ec929b1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,692 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174703692"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174703692"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174703692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174703692"}]},"ts":"1689174703692"} 2023-07-12 15:11:43,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-07-12 15:11:43,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure efc743968937366eb65d282b3ec929b1, server=jenkins-hbase20.apache.org,37993,1689174599377 in 179 msec 2023-07-12 15:11:43,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-07-12 15:11:43,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, ASSIGN in 338 msec 2023-07-12 15:11:43,696 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:11:43,696 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174703696"}]},"ts":"1689174703696"} 2023-07-12 15:11:43,697 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-07-12 15:11:43,699 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:11:43,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 387 msec 2023-07-12 15:11:43,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:43,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,829 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:43,829 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:43,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 15:11:43,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 15:11:43,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,830 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,831 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:43,831 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:43,831 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:43,831 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,831 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,832 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,832 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:43,832 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:43,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:43,833 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:43,833 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:43,833 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. started... 2023-07-12 15:11:43,834 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:11:43,835 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,835 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,835 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,835 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:43,835 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,835 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:43,842 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/d56b946920cc4f63bd79d5172592cdfa 2023-07-12 15:11:43,857 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/3545208c0c8a49baab5931eb1c09ae92 2023-07-12 15:11:43,862 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/d56b946920cc4f63bd79d5172592cdfa as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa 2023-07-12 15:11:43,869 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:43,870 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/3545208c0c8a49baab5931eb1c09ae92 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92 2023-07-12 15:11:43,875 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:43,876 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 42ms, sequenceid=6, compaction requested=false 2023-07-12 15:11:43,876 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:43,876 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:43,876 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:43,876 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:43,877 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:43,877 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,877 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:43,877 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:43,878 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:43,878 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,878 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:43,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:43,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:43,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 15:11:43,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b2f5894[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:43,882 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 15:11:43,882 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,882 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,882 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,882 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:43,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,883 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,883 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:43,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:43,883 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:43,884 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:43,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:43,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:43,884 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:43,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:43,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:43,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:43,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:43,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:11:43,888 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 15:11:43,889 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:43,889 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:43,889 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,889 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:43,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:43,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:43,891 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:43,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,891 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:43,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:43,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:43,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,892 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:43,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:43,894 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,895 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,907 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:43,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:43,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:43,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:43,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,908 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:43,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:43,907 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:43,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:43,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:43,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:43,909 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:43,909 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:43,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:43,908 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:43,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:43,909 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:43,909 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,909 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:43,910 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:43,910 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,910 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:43,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:43,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,000 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:11:44,009 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:44,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:44,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:44,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:44,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,022 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:44,022 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,022 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 15:11:44,022 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 15:11:44,023 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,023 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,024 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,024 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,024 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,024 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,024 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,024 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,025 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,025 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,025 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,025 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,025 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,026 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:44,027 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,027 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:44,027 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. started... 2023-07-12 15:11:44,027 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:44,438 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/39ae45c220c54e0c862db136b0bb24f9 2023-07-12 15:11:44,455 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/69d95cecde02457cbcf0709347043bf9 2023-07-12 15:11:44,459 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/39ae45c220c54e0c862db136b0bb24f9 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9 2023-07-12 15:11:44,463 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:44,464 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/69d95cecde02457cbcf0709347043bf9 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9 2023-07-12 15:11:44,467 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:44,468 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 441ms, sequenceid=19, compaction requested=false 2023-07-12 15:11:44,468 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:44,468 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:44,468 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:44,468 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:44,468 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,468 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,469 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,470 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,470 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,470 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,470 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 15:11:44,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@784b33c[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:44,471 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 15:11:44,472 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,472 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:44,472 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,472 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,472 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,473 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:44,472 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:44,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,473 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,474 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,474 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,474 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,474 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:44,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:11:44,476 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 15:11:44,476 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:44,476 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:44,476 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,476 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,477 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,478 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,478 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,478 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,480 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,480 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,480 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:44,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,481 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,482 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,483 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,483 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,483 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,582 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:44,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:44,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:44,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:44,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,604 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:44,604 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,605 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 15:11:44,605 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 15:11:44,605 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,605 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,606 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,606 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,606 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,606 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:44,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,608 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,608 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,609 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,609 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,609 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:44,609 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:44,609 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:44,609 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. started... 2023-07-12 15:11:44,609 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:44,609 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,610 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,610 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,610 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,611 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,611 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,621 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/e1c562d71f4148ca9b13bc653167d828 2023-07-12 15:11:44,633 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/d4f0f396598c49cb8660c93ca98a8e3b 2023-07-12 15:11:44,636 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/e1c562d71f4148ca9b13bc653167d828 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828 2023-07-12 15:11:44,639 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:44,640 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/d4f0f396598c49cb8660c93ca98a8e3b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b 2023-07-12 15:11:44,643 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:44,644 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 35ms, sequenceid=32, compaction requested=true 2023-07-12 15:11:44,644 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:44,644 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:44,644 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:44,644 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:44,644 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,644 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,644 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,645 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,645 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,645 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,645 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 15:11:44,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7435f9fc[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:44,647 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 15:11:44,647 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,647 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,647 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,647 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,648 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,648 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:44,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,648 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,649 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,649 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:44,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:11:44,651 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 15:11:44,651 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:44,651 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:44,651 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,651 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,653 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,653 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,653 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,654 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,654 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,654 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,656 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:44,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,656 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:44,657 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,757 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:44,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:44,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:44,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:44,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,778 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:44,778 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,778 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 15:11:44,778 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 15:11:44,779 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,779 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,780 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,780 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,780 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,780 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,781 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,781 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:44,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,783 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,783 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:44,784 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,784 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:44,784 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:44,784 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:44,784 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. started... 2023-07-12 15:11:44,784 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:44,784 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,785 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,785 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,785 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,785 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,785 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,796 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/5858a593a4634101858202e6a977138f 2023-07-12 15:11:44,812 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/ab32d04c43c14e5cb214c0bb009ebd12 2023-07-12 15:11:44,819 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/5858a593a4634101858202e6a977138f as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f 2023-07-12 15:11:44,824 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:44,825 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/ab32d04c43c14e5cb214c0bb009ebd12 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12 2023-07-12 15:11:44,831 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:44,832 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 48ms, sequenceid=45, compaction requested=true 2023-07-12 15:11:44,832 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:44,832 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:44,832 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:44,832 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:44,832 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,832 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:44,832 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:44,834 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,834 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,834 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,834 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 15:11:44,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24915047[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:44,838 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 15:11:44,838 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,844 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,844 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:44,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,845 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,844 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,845 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:44,845 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,845 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,845 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:44,845 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:44,845 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:44,846 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,846 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,846 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,847 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,847 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,847 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:44,847 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,847 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,847 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,847 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:44,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:11:44,854 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 15:11:44,854 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:44,854 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:44,854 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,854 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:44,860 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,860 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:44,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,860 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,860 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,860 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:44,861 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,861 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,861 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,862 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,863 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,866 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,867 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,867 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:44,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,878 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:44,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:44,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:44,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:44,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,878 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:44,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:44,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,879 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:44,979 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:44,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:45,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:45,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:45,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,003 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:45,003 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,003 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 15:11:45,003 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 15:11:45,004 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,004 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,005 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,005 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,005 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,005 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:45,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:45,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,007 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,007 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:45,007 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,007 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:45,007 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:45,007 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,007 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:45,007 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:45,007 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,007 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,007 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:45,007 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:45,008 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,007 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,008 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. started... 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:45,008 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:45,008 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:45,019 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/ae82b86646754c7f96deb199eb8e0288 2023-07-12 15:11:45,025 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,027 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,027 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,027 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:45,028 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,028 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:45,035 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/22fa376106a24b189da2e0b115c247d8 2023-07-12 15:11:45,041 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/ae82b86646754c7f96deb199eb8e0288 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288 2023-07-12 15:11:45,048 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:45,049 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/22fa376106a24b189da2e0b115c247d8 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8 2023-07-12 15:11:45,056 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:45,057 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 49ms, sequenceid=58, compaction requested=true 2023-07-12 15:11:45,057 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:45,057 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:45,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:45,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:45,057 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:45,058 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,058 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:45,059 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,059 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:45,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 15:11:45,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@653b47b9[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:45,061 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 15:11:45,061 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,061 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:45,061 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:45,061 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,061 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:45,061 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:45,061 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:45,063 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,062 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,063 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:45,062 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:45,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:11:45,065 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 15:11:45,065 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:45,065 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:45,065 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,065 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:45,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,067 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,067 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:45,068 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,068 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,069 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,077 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,078 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:45,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,077 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,078 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,078 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,078 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:45,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,078 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,079 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,178 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:45,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:45,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:45,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:45,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,203 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:45,203 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,203 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-12 15:11:45,203 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-12 15:11:45,204 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,204 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,205 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,205 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,205 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,206 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:45,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:45,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,208 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,208 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:45,208 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,208 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,208 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:45,208 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:45,208 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:45,208 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:45,209 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-12 15:11:45,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:45,209 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:45,209 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,210 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. started... 2023-07-12 15:11:45,210 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:45,211 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,211 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,211 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,211 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:45,212 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,212 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:45,220 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/2057df08accf4d2093d62f2aa672132b 2023-07-12 15:11:45,235 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/05599c3b00794f03a1519c7e6adbc193 2023-07-12 15:11:45,240 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/2057df08accf4d2093d62f2aa672132b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b 2023-07-12 15:11:45,246 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:45,247 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/05599c3b00794f03a1519c7e6adbc193 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193 2023-07-12 15:11:45,251 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:45,252 INFO [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 42ms, sequenceid=71, compaction requested=true 2023-07-12 15:11:45,252 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:45,252 DEBUG [rs(jenkins-hbase20.apache.org,37993,1689174599377)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,253 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:45,253 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:45,253 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,253 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-12 15:11:45,253 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-12 15:11:45,254 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,254 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:45,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:45,255 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,255 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:45,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:45,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:45,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:45,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-12 15:11:45,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47840571[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:45,258 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-12 15:11:45,258 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:45,261 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:45,261 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,261 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,261 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:45,262 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,263 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:45,263 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:45,263 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,263 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,263 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:45,263 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-12 15:11:45,264 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:45,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:45,264 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-12 15:11:45,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:45,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:45,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:45,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,37993,1689174599377': 2023-07-12 15:11:45,269 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-12 15:11:45,269 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:45,269 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:45,269 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-12 15:11:45,269 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:45,283 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,283 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,283 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,283 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,283 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,284 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,284 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,285 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:45,286 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:45,287 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,290 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,291 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:45,291 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,295 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,295 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,295 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,295 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,295 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,296 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:45,295 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-12 15:11:45,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:45,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:45,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:45,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:45,297 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,297 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,297 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:45,297 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,297 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-12 15:11:45,397 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:45,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:45,413 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,414 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=efc743968937366eb65d282b3ec929b1, daughterA=913a12b2aa6b4c0f26656830060f788e, daughterB=7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:45,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=efc743968937366eb65d282b3ec929b1, daughterA=913a12b2aa6b4c0f26656830060f788e, daughterB=7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:45,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=efc743968937366eb65d282b3ec929b1, daughterA=913a12b2aa6b4c0f26656830060f788e, daughterB=7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:45,416 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=efc743968937366eb65d282b3ec929b1, daughterA=913a12b2aa6b4c0f26656830060f788e, daughterB=7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:45,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, UNASSIGN}] 2023-07-12 15:11:45,418 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, UNASSIGN 2023-07-12 15:11:45,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=efc743968937366eb65d282b3ec929b1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174705419"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174705419"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174705419"}]},"ts":"1689174705419"} 2023-07-12 15:11:45,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure efc743968937366eb65d282b3ec929b1, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:45,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing efc743968937366eb65d282b3ec929b1, disabling compactions & flushes 2023-07-12 15:11:45,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. after waiting 0 ms 2023-07-12 15:11:45,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,574 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing efc743968937366eb65d282b3ec929b1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:11:45,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/ac7c326025b94b2c8aed4a7daf6f3758 2023-07-12 15:11:45,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/b53c6891815e48a5b999f2a7f1ecc671 2023-07-12 15:11:45,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf1/ac7c326025b94b2c8aed4a7daf6f3758 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758 2023-07-12 15:11:45,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:45,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.tmp/cf2/b53c6891815e48a5b999f2a7f1ecc671 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671 2023-07-12 15:11:45,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:45,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for efc743968937366eb65d282b3ec929b1 in 50ms, sequenceid=82, compaction requested=true 2023-07-12 15:11:45,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:11:45,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:45,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. 2023-07-12 15:11:45,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for efc743968937366eb65d282b3ec929b1: 2023-07-12 15:11:45,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=efc743968937366eb65d282b3ec929b1, regionState=CLOSED 2023-07-12 15:11:45,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174705632"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174705632"}]},"ts":"1689174705632"} 2023-07-12 15:11:45,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-07-12 15:11:45,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure efc743968937366eb65d282b3ec929b1, server=jenkins-hbase20.apache.org,37993,1689174599377 in 213 msec 2023-07-12 15:11:45,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-07-12 15:11:45,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=efc743968937366eb65d282b3ec929b1, UNASSIGN in 219 msec 2023-07-12 15:11:45,650 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=efc743968937366eb65d282b3ec929b1, threads=14 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,651 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,655 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92, top=false 2023-07-12 15:11:45,656 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288, top=false 2023-07-12 15:11:45,656 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b, top=false 2023-07-12 15:11:45,659 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8, top=false 2023-07-12 15:11:45,660 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9, top=false 2023-07-12 15:11:45,661 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b, top=false 2023-07-12 15:11:45,661 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12, top=false 2023-07-12 15:11:45,662 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828, top=false 2023-07-12 15:11:45,662 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671, top=false 2023-07-12 15:11:45,664 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9, top=false 2023-07-12 15:11:45,665 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,665 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa, top=false 2023-07-12 15:11:45,667 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f, top=false 2023-07-12 15:11:45,669 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193, top=false 2023-07-12 15:11:45,672 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758, top=false 2023-07-12 15:11:45,672 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,695 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,695 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,695 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,695 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,695 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,695 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,699 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,700 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,700 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,700 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,703 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,704 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,704 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/.splits/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193 for child: 913a12b2aa6b4c0f26656830060f788e, parent: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,706 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,706 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,706 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,707 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,707 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,708 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,708 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,709 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,709 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,710 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,710 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,711 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,711 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193 for region: efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:45,711 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region efc743968937366eb65d282b3ec929b1 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-07-12 15:11:45,732 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:11:45,733 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:11:45,734 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1689174705734"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1689174705734"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1689174705734"}]},"ts":"1689174705734"} 2023-07-12 15:11:45,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174705734"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174705734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174705734"}]},"ts":"1689174705734"} 2023-07-12 15:11:45,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174705734"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174705734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174705734"}]},"ts":"1689174705734"} 2023-07-12 15:11:45,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, ASSIGN}] 2023-07-12 15:11:45,738 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, ASSIGN 2023-07-12 15:11:45,739 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:11:45,739 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, ASSIGN 2023-07-12 15:11:45,740 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,37993,1689174599377; forceNewPlan=false, retain=false 2023-07-12 15:11:45,889 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:11:45,891 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=7e2b5ead78aa72ddef1d8723f8d1010d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,891 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174705891"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174705891"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174705891"}]},"ts":"1689174705891"} 2023-07-12 15:11:45,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 7e2b5ead78aa72ddef1d8723f8d1010d, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:45,894 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=913a12b2aa6b4c0f26656830060f788e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:45,894 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174705894"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174705894"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174705894"}]},"ts":"1689174705894"} 2023-07-12 15:11:45,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 913a12b2aa6b4c0f26656830060f788e, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:46,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7e2b5ead78aa72ddef1d8723f8d1010d, NAME => 'testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.', STARTKEY => '71', ENDKEY => ''} 2023-07-12 15:11:46,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:46,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:46,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,052 INFO [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,054 DEBUG [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/cf1 2023-07-12 15:11:46,054 DEBUG [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/cf1 2023-07-12 15:11:46,055 INFO [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e2b5ead78aa72ddef1d8723f8d1010d columnFamilyName cf1 2023-07-12 15:11:46,055 INFO [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] regionserver.HStore(310): Store=7e2b5ead78aa72ddef1d8723f8d1010d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:46,056 INFO [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,058 DEBUG [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/cf2 2023-07-12 15:11:46,058 DEBUG [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/cf2 2023-07-12 15:11:46,058 INFO [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e2b5ead78aa72ddef1d8723f8d1010d columnFamilyName cf2 2023-07-12 15:11:46,059 INFO [StoreOpener-7e2b5ead78aa72ddef1d8723f8d1010d-1] regionserver.HStore(310): Store=7e2b5ead78aa72ddef1d8723f8d1010d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:46,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:46,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 7e2b5ead78aa72ddef1d8723f8d1010d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59200287, jitterRate=-0.11784698069095612}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:46,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 7e2b5ead78aa72ddef1d8723f8d1010d: 2023-07-12 15:11:46,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d., pid=133, masterSystemTime=1689174706043 2023-07-12 15:11:46,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,071 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=7e2b5ead78aa72ddef1d8723f8d1010d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:46,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 913a12b2aa6b4c0f26656830060f788e, NAME => 'testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.', STARTKEY => '', ENDKEY => '71'} 2023-07-12 15:11:46,072 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174706071"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174706071"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174706071"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174706071"}]},"ts":"1689174706071"} 2023-07-12 15:11:46,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:46,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:46,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,073 INFO [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-07-12 15:11:46,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 7e2b5ead78aa72ddef1d8723f8d1010d, server=jenkins-hbase20.apache.org,37993,1689174599377 in 180 msec 2023-07-12 15:11:46,074 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1 2023-07-12 15:11:46,074 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1 2023-07-12 15:11:46,074 INFO [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 913a12b2aa6b4c0f26656830060f788e columnFamilyName cf1 2023-07-12 15:11:46,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, ASSIGN in 337 msec 2023-07-12 15:11:46,079 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b 2023-07-12 15:11:46,081 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9 2023-07-12 15:11:46,084 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f 2023-07-12 15:11:46,086 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758 2023-07-12 15:11:46,089 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288 2023-07-12 15:11:46,092 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa 2023-07-12 15:11:46,094 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828 2023-07-12 15:11:46,094 INFO [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(310): Store=913a12b2aa6b4c0f26656830060f788e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:46,094 INFO [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,095 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2 2023-07-12 15:11:46,095 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2 2023-07-12 15:11:46,095 INFO [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 913a12b2aa6b4c0f26656830060f788e columnFamilyName cf2 2023-07-12 15:11:46,100 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193 2023-07-12 15:11:46,103 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8 2023-07-12 15:11:46,106 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92 2023-07-12 15:11:46,109 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9 2023-07-12 15:11:46,112 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12 2023-07-12 15:11:46,114 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671 2023-07-12 15:11:46,117 DEBUG [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b 2023-07-12 15:11:46,117 INFO [StoreOpener-913a12b2aa6b4c0f26656830060f788e-1] regionserver.HStore(310): Store=913a12b2aa6b4c0f26656830060f788e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:46,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:46,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 913a12b2aa6b4c0f26656830060f788e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65339976, jitterRate=-0.026358485221862793}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:46,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 913a12b2aa6b4c0f26656830060f788e: 2023-07-12 15:11:46,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e., pid=134, masterSystemTime=1689174706043 2023-07-12 15:11:46,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:46,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:11:46,123 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:46,123 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:46,123 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 15:11:46,123 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-12 15:11:46,123 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1912): 913a12b2aa6b4c0f26656830060f788e/cf2 is initiating minor compaction (all files) 2023-07-12 15:11:46,123 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1912): 913a12b2aa6b4c0f26656830060f788e/cf1 is initiating minor compaction (all files) 2023-07-12 15:11:46,123 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 913a12b2aa6b4c0f26656830060f788e/cf2 in testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,123 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 913a12b2aa6b4c0f26656830060f788e/cf1 in testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,124 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/.tmp, totalSize=34.9 K 2023-07-12 15:11:46,124 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/.tmp, totalSize=34.9 K 2023-07-12 15:11:46,124 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174703818 2023-07-12 15:11:46,124 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174703818 2023-07-12 15:11:46,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,124 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174704011 2023-07-12 15:11:46,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,125 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174704011 2023-07-12 15:11:46,125 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174704586 2023-07-12 15:11:46,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=913a12b2aa6b4c0f26656830060f788e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:46,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174706125"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174706125"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174706125"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174706125"}]},"ts":"1689174706125"} 2023-07-12 15:11:46,125 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174704759 2023-07-12 15:11:46,125 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174704586 2023-07-12 15:11:46,125 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174704984 2023-07-12 15:11:46,125 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174704759 2023-07-12 15:11:46,126 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174705182 2023-07-12 15:11:46,126 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174704984 2023-07-12 15:11:46,126 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174705401 2023-07-12 15:11:46,126 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174705182 2023-07-12 15:11:46,127 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174705401 2023-07-12 15:11:46,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-07-12 15:11:46,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 913a12b2aa6b4c0f26656830060f788e, server=jenkins-hbase20.apache.org,37993,1689174599377 in 231 msec 2023-07-12 15:11:46,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-07-12 15:11:46,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, ASSIGN in 392 msec 2023-07-12 15:11:46,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=efc743968937366eb65d282b3ec929b1, daughterA=913a12b2aa6b4c0f26656830060f788e, daughterB=7e2b5ead78aa72ddef1d8723f8d1010d in 717 msec 2023-07-12 15:11:46,143 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] throttle.PressureAwareThroughputController(145): 913a12b2aa6b4c0f26656830060f788e#cf2#compaction#125 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:46,147 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] throttle.PressureAwareThroughputController(145): 913a12b2aa6b4c0f26656830060f788e#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:46,154 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/.tmp/cf2/67c4f4c3cd5944bf9866a86c0bc53453 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/67c4f4c3cd5944bf9866a86c0bc53453 2023-07-12 15:11:46,155 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/.tmp/cf1/828cb9d873ea45648732237c9fcaea64 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/828cb9d873ea45648732237c9fcaea64 2023-07-12 15:11:46,157 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 913a12b2aa6b4c0f26656830060f788e/cf2 of 913a12b2aa6b4c0f26656830060f788e into 67c4f4c3cd5944bf9866a86c0bc53453(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:46,157 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 913a12b2aa6b4c0f26656830060f788e: 2023-07-12 15:11:46,157 INFO [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e., storeName=913a12b2aa6b4c0f26656830060f788e/cf2, priority=9, startTime=1689174706123; duration=0sec 2023-07-12 15:11:46,157 DEBUG [RS:2;jenkins-hbase20:37993-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:46,159 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 913a12b2aa6b4c0f26656830060f788e/cf1 of 913a12b2aa6b4c0f26656830060f788e into 828cb9d873ea45648732237c9fcaea64(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:46,159 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 913a12b2aa6b4c0f26656830060f788e: 2023-07-12 15:11:46,159 INFO [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e., storeName=913a12b2aa6b4c0f26656830060f788e/cf1, priority=9, startTime=1689174706122; duration=0sec 2023-07-12 15:11:46,159 DEBUG [RS:2;jenkins-hbase20:37993-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:46,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37993] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:40686 deadline: 1689174766165, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1. is not online on jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:46,281 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-07-12 15:11:46,285 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:46,287 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-12 15:11:46,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174706292"}]},"ts":"1689174706292"} 2023-07-12 15:11:46,294 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-07-12 15:11:46,296 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-07-12 15:11:46,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, UNASSIGN}] 2023-07-12 15:11:46,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, UNASSIGN 2023-07-12 15:11:46,298 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=7e2b5ead78aa72ddef1d8723f8d1010d, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:46,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174706298"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174706298"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174706298"}]},"ts":"1689174706298"} 2023-07-12 15:11:46,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 7e2b5ead78aa72ddef1d8723f8d1010d, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:46,301 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, UNASSIGN 2023-07-12 15:11:46,302 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=913a12b2aa6b4c0f26656830060f788e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:46,302 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174706302"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174706302"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174706302"}]},"ts":"1689174706302"} 2023-07-12 15:11:46,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 913a12b2aa6b4c0f26656830060f788e, server=jenkins-hbase20.apache.org,37993,1689174599377}] 2023-07-12 15:11:46,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-12 15:11:46,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 7e2b5ead78aa72ddef1d8723f8d1010d, disabling compactions & flushes 2023-07-12 15:11:46,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. after waiting 0 ms 2023-07-12 15:11:46,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-07-12 15:11:46,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:46,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d. 2023-07-12 15:11:46,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 7e2b5ead78aa72ddef1d8723f8d1010d: 2023-07-12 15:11:46,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 913a12b2aa6b4c0f26656830060f788e, disabling compactions & flushes 2023-07-12 15:11:46,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. after waiting 0 ms 2023-07-12 15:11:46,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,467 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=7e2b5ead78aa72ddef1d8723f8d1010d, regionState=CLOSED 2023-07-12 15:11:46,467 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174706467"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174706467"}]},"ts":"1689174706467"} 2023-07-12 15:11:46,468 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758] to archive 2023-07-12 15:11:46,469 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:46,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-07-12 15:11:46,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 7e2b5ead78aa72ddef1d8723f8d1010d, server=jenkins-hbase20.apache.org,37993,1689174599377 in 170 msec 2023-07-12 15:11:46,471 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d56b946920cc4f63bd79d5172592cdfa 2023-07-12 15:11:46,473 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-39ae45c220c54e0c862db136b0bb24f9 2023-07-12 15:11:46,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7e2b5ead78aa72ddef1d8723f8d1010d, UNASSIGN in 173 msec 2023-07-12 15:11:46,474 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-e1c562d71f4148ca9b13bc653167d828 2023-07-12 15:11:46,476 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-5858a593a4634101858202e6a977138f 2023-07-12 15:11:46,478 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ae82b86646754c7f96deb199eb8e0288 2023-07-12 15:11:46,479 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-2057df08accf4d2093d62f2aa672132b 2023-07-12 15:11:46,481 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ac7c326025b94b2c8aed4a7daf6f3758 2023-07-12 15:11:46,483 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671] to archive 2023-07-12 15:11:46,484 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:46,486 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-3545208c0c8a49baab5931eb1c09ae92 2023-07-12 15:11:46,487 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-69d95cecde02457cbcf0709347043bf9 2023-07-12 15:11:46,489 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-d4f0f396598c49cb8660c93ca98a8e3b 2023-07-12 15:11:46,491 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-ab32d04c43c14e5cb214c0bb009ebd12 2023-07-12 15:11:46,492 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-22fa376106a24b189da2e0b115c247d8 2023-07-12 15:11:46,494 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-05599c3b00794f03a1519c7e6adbc193 2023-07-12 15:11:46,496 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/testSplitTableRegionEmptyDaughter=efc743968937366eb65d282b3ec929b1-b53c6891815e48a5b999f2a7f1ecc671 2023-07-12 15:11:46,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:11:46,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:46,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e. 2023-07-12 15:11:46,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 913a12b2aa6b4c0f26656830060f788e: 2023-07-12 15:11:46,503 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,504 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=913a12b2aa6b4c0f26656830060f788e, regionState=CLOSED 2023-07-12 15:11:46,504 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1689174706504"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174706504"}]},"ts":"1689174706504"} 2023-07-12 15:11:46,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-07-12 15:11:46,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 913a12b2aa6b4c0f26656830060f788e, server=jenkins-hbase20.apache.org,37993,1689174599377 in 203 msec 2023-07-12 15:11:46,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-07-12 15:11:46,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=913a12b2aa6b4c0f26656830060f788e, UNASSIGN in 212 msec 2023-07-12 15:11:46,512 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174706512"}]},"ts":"1689174706512"} 2023-07-12 15:11:46,513 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-07-12 15:11:46,515 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-07-12 15:11:46,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 227 msec 2023-07-12 15:11:46,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-12 15:11:46,597 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-07-12 15:11:46,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,604 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,605 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-12 15:11:46,612 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:46,612 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,612 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,615 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/recovered.edits] 2023-07-12 15:11:46,615 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/recovered.edits] 2023-07-12 15:11:46,615 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/recovered.edits] 2023-07-12 15:11:46,617 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/2057df08accf4d2093d62f2aa672132b 2023-07-12 15:11:46,617 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/828cb9d873ea45648732237c9fcaea64 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf1/828cb9d873ea45648732237c9fcaea64 2023-07-12 15:11:46,618 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/39ae45c220c54e0c862db136b0bb24f9 2023-07-12 15:11:46,618 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/recovered.edits/88.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d/recovered.edits/88.seqid 2023-07-12 15:11:46,619 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/67c4f4c3cd5944bf9866a86c0bc53453 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/cf2/67c4f4c3cd5944bf9866a86c0bc53453 2023-07-12 15:11:46,619 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/5858a593a4634101858202e6a977138f 2023-07-12 15:11:46,619 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/7e2b5ead78aa72ddef1d8723f8d1010d 2023-07-12 15:11:46,620 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ac7c326025b94b2c8aed4a7daf6f3758 2023-07-12 15:11:46,620 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/ae82b86646754c7f96deb199eb8e0288 2023-07-12 15:11:46,620 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e/recovered.edits/90.seqid 2023-07-12 15:11:46,621 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/913a12b2aa6b4c0f26656830060f788e 2023-07-12 15:11:46,621 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/d56b946920cc4f63bd79d5172592cdfa 2023-07-12 15:11:46,622 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf1/e1c562d71f4148ca9b13bc653167d828 2023-07-12 15:11:46,624 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/05599c3b00794f03a1519c7e6adbc193 2023-07-12 15:11:46,625 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/22fa376106a24b189da2e0b115c247d8 2023-07-12 15:11:46,626 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/3545208c0c8a49baab5931eb1c09ae92 2023-07-12 15:11:46,627 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/69d95cecde02457cbcf0709347043bf9 2023-07-12 15:11:46,628 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/ab32d04c43c14e5cb214c0bb009ebd12 2023-07-12 15:11:46,629 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/b53c6891815e48a5b999f2a7f1ecc671 2023-07-12 15:11:46,630 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/cf2/d4f0f396598c49cb8660c93ca98a8e3b 2023-07-12 15:11:46,632 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/recovered.edits/85.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1/recovered.edits/85.seqid 2023-07-12 15:11:46,633 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegionEmptyDaughter/efc743968937366eb65d282b3ec929b1 2023-07-12 15:11:46,633 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-12 15:11:46,635 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,636 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-07-12 15:11:46,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-07-12 15:11:46,640 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,640 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-07-12 15:11:46,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174706640"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:46,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174706640"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:46,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174706640"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:46,642 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:11:46,642 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => efc743968937366eb65d282b3ec929b1, NAME => 'testSplitTableRegionEmptyDaughter,,1689174703312.efc743968937366eb65d282b3ec929b1.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 913a12b2aa6b4c0f26656830060f788e, NAME => 'testSplitTableRegionEmptyDaughter,,1689174705414.913a12b2aa6b4c0f26656830060f788e.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 7e2b5ead78aa72ddef1d8723f8d1010d, NAME => 'testSplitTableRegionEmptyDaughter,71,1689174705414.7e2b5ead78aa72ddef1d8723f8d1010d.', STARTKEY => '71', ENDKEY => ''}] 2023-07-12 15:11:46,642 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-07-12 15:11:46,642 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174706642"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:46,644 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-07-12 15:11:46,645 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-12 15:11:46,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 46 msec 2023-07-12 15:11:46,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-12 15:11:46,707 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-07-12 15:11:46,728 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376 (was 377), OpenFileDescriptor=842 (was 844), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=98 (was 98), ProcessCount=168 (was 168), AvailableMemoryMB=5305 (was 5309) 2023-07-12 15:11:46,740 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=98, ProcessCount=168, AvailableMemoryMB=5305 2023-07-12 15:11:46,740 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:46,742 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-12 15:11:46,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-07-12 15:11:46,743 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-07-12 15:11:46,745 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-07-12 15:11:46,745 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-12 15:11:46,747 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:46,747 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 empty. 2023-07-12 15:11:46,747 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:46,747 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-12 15:11:46,757 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-07-12 15:11:46,758 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => c9e02faa319de90816e4c0f2fab9deb6, NAME => 'testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp 2023-07-12 15:11:46,767 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:46,767 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing c9e02faa319de90816e4c0f2fab9deb6, disabling compactions & flushes 2023-07-12 15:11:46,767 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:46,767 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:46,767 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. after waiting 0 ms 2023-07-12 15:11:46,767 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:46,767 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:46,767 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:46,770 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-07-12 15:11:46,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174706771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174706771"}]},"ts":"1689174706771"} 2023-07-12 15:11:46,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-12 15:11:46,773 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-12 15:11:46,773 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174706773"}]},"ts":"1689174706773"} 2023-07-12 15:11:46,774 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-07-12 15:11:46,777 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-07-12 15:11:46,778 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-12 15:11:46,779 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-12 15:11:46,779 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-12 15:11:46,779 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-12 15:11:46,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, ASSIGN}] 2023-07-12 15:11:46,781 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, ASSIGN 2023-07-12 15:11:46,781 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:11:46,932 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-12 15:11:46,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=c9e02faa319de90816e4c0f2fab9deb6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:46,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174706934"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174706934"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174706934"}]},"ts":"1689174706934"} 2023-07-12 15:11:46,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure c9e02faa319de90816e4c0f2fab9deb6, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:47,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c9e02faa319de90816e4c0f2fab9deb6, NAME => 'testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.', STARTKEY => '', ENDKEY => ''} 2023-07-12 15:11:47,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:47,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:47,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,107 INFO [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,110 DEBUG [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1 2023-07-12 15:11:47,110 DEBUG [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1 2023-07-12 15:11:47,110 INFO [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9e02faa319de90816e4c0f2fab9deb6 columnFamilyName cf1 2023-07-12 15:11:47,111 INFO [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] regionserver.HStore(310): Store=c9e02faa319de90816e4c0f2fab9deb6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:47,111 INFO [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,114 DEBUG [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2 2023-07-12 15:11:47,114 DEBUG [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2 2023-07-12 15:11:47,114 INFO [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9e02faa319de90816e4c0f2fab9deb6 columnFamilyName cf2 2023-07-12 15:11:47,115 INFO [StoreOpener-c9e02faa319de90816e4c0f2fab9deb6-1] regionserver.HStore(310): Store=c9e02faa319de90816e4c0f2fab9deb6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:47,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:47,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:47,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-12 15:11:47,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened c9e02faa319de90816e4c0f2fab9deb6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62178139, jitterRate=-0.07347352802753448}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:47,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:47,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6., pid=143, masterSystemTime=1689174707091 2023-07-12 15:11:47,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,123 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=c9e02faa319de90816e4c0f2fab9deb6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,123 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174707123"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174707123"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174707123"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174707123"}]},"ts":"1689174707123"} 2023-07-12 15:11:47,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-07-12 15:11:47,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure c9e02faa319de90816e4c0f2fab9deb6, server=jenkins-hbase20.apache.org,41741,1689174599302 in 186 msec 2023-07-12 15:11:47,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-07-12 15:11:47,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, ASSIGN in 347 msec 2023-07-12 15:11:47,128 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-12 15:11:47,128 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174707128"}]},"ts":"1689174707128"} 2023-07-12 15:11:47,129 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-07-12 15:11:47,130 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-07-12 15:11:47,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 387 msec 2023-07-12 15:11:47,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:47,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 15:11:47,255 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 15:11:47,255 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,255 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 15:11:47,255 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 15:11:47,255 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,255 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,256 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,256 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,256 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,256 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,257 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,257 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,257 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,258 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,257 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:47,258 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:47,258 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,258 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. started... 2023-07-12 15:11:47,259 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-12 15:11:47,259 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,260 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,260 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,260 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,260 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,260 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,266 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/f041fd77d7194f809eb8ec23a62c32d6 2023-07-12 15:11:47,276 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/75e58a986e30408ca0390d3f68603b74 2023-07-12 15:11:47,280 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/f041fd77d7194f809eb8ec23a62c32d6 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6 2023-07-12 15:11:47,285 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:47,286 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/75e58a986e30408ca0390d3f68603b74 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74 2023-07-12 15:11:47,290 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:47,291 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 31ms, sequenceid=6, compaction requested=false 2023-07-12 15:11:47,291 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-07-12 15:11:47,291 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:47,291 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,291 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:47,291 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:47,291 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,291 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,291 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,292 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,292 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,293 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,293 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 15:11:47,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c25fe64[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:47,295 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 15:11:47,295 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,296 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,297 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,297 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,298 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,298 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:47,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 15:11:47,301 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 15:11:47,301 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:47,301 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:47,301 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 15:11:47,301 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,302 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,303 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,303 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,304 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,306 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,307 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,307 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 15:11:47,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:47,310 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,311 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:47,311 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,310 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,311 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,311 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,312 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,312 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,312 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,312 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,312 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,312 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,381 ERROR [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid 5606ee4f-c045-409e-a742-091e025e583c) service to localhost.localdomain/127.0.0.1:34607 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:11:47,411 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:47,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:47,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:47,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:47,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 15:11:47,433 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 15:11:47,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 15:11:47,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 15:11:47,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,433 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,434 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,434 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,434 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,434 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,435 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,435 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:47,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,437 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,437 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,438 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,439 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,439 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,442 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:47,443 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:47,442 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,443 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,443 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,443 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. started... 2023-07-12 15:11:47,443 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,443 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,443 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,443 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:47,450 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,450 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,450 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,456 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/cb3e3368115142f78fcdfcbcae869e23 2023-07-12 15:11:47,470 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/4ad5389de4614580a29418a35865abec 2023-07-12 15:11:47,473 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/cb3e3368115142f78fcdfcbcae869e23 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23 2023-07-12 15:11:47,476 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:47,477 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/4ad5389de4614580a29418a35865abec as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec 2023-07-12 15:11:47,479 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec, entries=10, sequenceid=19, filesize=5.0 K 2023-07-12 15:11:47,480 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 37ms, sequenceid=19, compaction requested=false 2023-07-12 15:11:47,480 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:47,480 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,480 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:47,480 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:47,480 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,480 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,480 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,481 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,481 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,482 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,482 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 15:11:47,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@329896cf[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:47,483 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 15:11:47,483 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:47,484 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,484 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:47,484 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:47,485 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,485 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,486 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,486 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:47,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 15:11:47,488 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 15:11:47,488 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:47,488 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:47,488 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 15:11:47,488 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,490 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,489 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,490 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,491 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,492 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,493 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,493 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,496 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 15:11:47,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:47,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,496 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:47,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,497 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,597 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:47,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:47,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:47,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:47,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 15:11:47,615 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 15:11:47,615 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,615 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 15:11:47,615 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 15:11:47,616 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,616 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,617 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,617 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,617 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,617 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,617 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,617 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:47,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,618 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,618 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:47,619 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:47,619 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,619 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. started... 2023-07-12 15:11:47,620 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,620 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:47,620 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,620 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,620 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,621 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,621 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,764 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/e0208a161cc34887baf5463e3cde00a9 2023-07-12 15:11:47,775 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/0dd0d393a0944bc5904f0d0195d07a4b 2023-07-12 15:11:47,779 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/e0208a161cc34887baf5463e3cde00a9 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9 2023-07-12 15:11:47,782 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:47,783 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/0dd0d393a0944bc5904f0d0195d07a4b as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b 2023-07-12 15:11:47,786 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b, entries=10, sequenceid=32, filesize=5.0 K 2023-07-12 15:11:47,787 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 167ms, sequenceid=32, compaction requested=true 2023-07-12 15:11:47,787 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:47,787 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,787 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:47,787 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:47,787 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,787 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,787 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,788 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,788 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,789 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,789 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,791 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 15:11:47,791 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d2119aa[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:47,791 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 15:11:47,791 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,791 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,791 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,791 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,792 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,792 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:47,792 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,793 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,793 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:47,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 15:11:47,797 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 15:11:47,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:47,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:47,797 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 15:11:47,797 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,799 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,798 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,799 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,800 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,802 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,802 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,802 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 15:11:47,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:47,812 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,812 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:47,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:47,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,813 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,913 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:47,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:47,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:47,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:47,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 15:11:47,936 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 15:11:47,936 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,936 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 15:11:47,936 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 15:11:47,937 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,937 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,938 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,938 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,938 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:47,938 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,939 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,939 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:47,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:47,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,941 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:47,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:47,942 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,941 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:47,942 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,942 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:47,942 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:47,942 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,942 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,942 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. started... 2023-07-12 15:11:47,942 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,942 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:47,943 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,943 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,943 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,943 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,943 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,943 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,950 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/b66a4b3dbdc84966905ec1db44042e87 2023-07-12 15:11:47,963 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/44f2a36ef5484441b8c2b40bd8cb85eb 2023-07-12 15:11:47,966 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/b66a4b3dbdc84966905ec1db44042e87 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87 2023-07-12 15:11:47,970 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:47,970 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/44f2a36ef5484441b8c2b40bd8cb85eb as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb 2023-07-12 15:11:47,974 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb, entries=10, sequenceid=45, filesize=5.0 K 2023-07-12 15:11:47,975 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 33ms, sequenceid=45, compaction requested=true 2023-07-12 15:11:47,975 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:47,975 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:47,975 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:47,975 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:47,975 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,975 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:47,975 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:47,976 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,977 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,977 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,977 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:47,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 15:11:47,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a692af2[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:47,979 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 15:11:47,979 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,980 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,980 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:47,980 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:47,981 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:47,981 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,981 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,981 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,981 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,981 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,982 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,982 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,983 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,983 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:47,983 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:47,986 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 15:11:47,986 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 15:11:47,986 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:47,986 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:47,986 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 15:11:47,986 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:47,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:47,995 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:47,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,996 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,996 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,998 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:47,998 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:47,999 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,000 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,000 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,000 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,000 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,000 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,000 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 15:11:48,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:48,001 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,001 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,102 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:48,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:48,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:48,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:48,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 15:11:48,126 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 15:11:48,126 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,126 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 15:11:48,126 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 15:11:48,127 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,127 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,128 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,128 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,128 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,128 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,129 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,129 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:48,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:48,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:48,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:48,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:48,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:48,131 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:48,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:48,132 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:48,133 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:48,133 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:48,133 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:48,133 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,133 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. started... 2023-07-12 15:11:48,133 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:48,142 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,144 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,144 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,144 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:48,145 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,145 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:48,146 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/2f4846d1be544d7392b5c386c4a905ea 2023-07-12 15:11:48,158 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/ed61459025724feeae7259c340597e68 2023-07-12 15:11:48,161 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/2f4846d1be544d7392b5c386c4a905ea as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea 2023-07-12 15:11:48,165 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:48,166 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/ed61459025724feeae7259c340597e68 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68 2023-07-12 15:11:48,170 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68, entries=10, sequenceid=58, filesize=5.0 K 2023-07-12 15:11:48,171 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 38ms, sequenceid=58, compaction requested=true 2023-07-12 15:11:48,171 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:48,171 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,171 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:48,171 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:48,171 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,171 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:48,171 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:48,172 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,172 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:48,172 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,172 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:48,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 15:11:48,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@590c2f05[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:48,174 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 15:11:48,174 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,174 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,174 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,174 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,175 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,175 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:48,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:48,175 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,176 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:48,176 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:48,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 15:11:48,178 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 15:11:48,178 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:48,178 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:48,178 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 15:11:48,178 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:48,178 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,178 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,178 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,179 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,178 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:48,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,179 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,179 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:48,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,181 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,182 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:48,183 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,183 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,184 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,191 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,192 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 15:11:48,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,192 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,193 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,293 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:48,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:48,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-12 15:11:48,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-12 15:11:48,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-12 15:11:48,305 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-12 15:11:48,306 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,306 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-12 15:11:48,306 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-12 15:11:48,306 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,306 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,307 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,307 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,307 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,307 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,307 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,307 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-12 15:11:48,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:48,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:48,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,33703,1689174599345' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-12 15:11:48,309 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-12 15:11:48,308 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37993,1689174599377' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:48,309 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-12 15:11:48,309 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-12 15:11:48,309 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-12 15:11:48,309 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,309 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. started... 2023-07-12 15:11:48,309 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-12 15:11:48,310 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,310 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,310 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,310 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:48,310 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,311 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:48,316 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc 2023-07-12 15:11:48,325 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/f361910f5fbc418a9a2135f1baead486 2023-07-12 15:11:48,329 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc 2023-07-12 15:11:48,331 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:48,332 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/f361910f5fbc418a9a2135f1baead486 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486 2023-07-12 15:11:48,334 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486, entries=10, sequenceid=71, filesize=5.0 K 2023-07-12 15:11:48,335 INFO [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 26ms, sequenceid=71, compaction requested=true 2023-07-12 15:11:48,335 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:48,335 DEBUG [rs(jenkins-hbase20.apache.org,41741,1689174599302)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,335 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-12 15:11:48,335 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-12 15:11:48,335 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,335 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-12 15:11:48,335 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-12 15:11:48,344 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,344 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:48,345 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,345 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-12 15:11:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:48,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:48,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41741,1689174599302' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-12 15:11:48,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fd4f405[Count = 0] remaining members to acquire global barrier 2023-07-12 15:11:48,347 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-12 15:11:48,347 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,358 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-12 15:11:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,358 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:48,358 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37993,1689174599377' in zk 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41741,1689174599302' in zk 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-12 15:11:48,359 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,33703,1689174599345' in zk 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,37993,1689174599377' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,361 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,41741,1689174599302' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:48,361 DEBUG [member: 'jenkins-hbase20.apache.org,33703,1689174599345' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-12 15:11:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41741,1689174599302': 2023-07-12 15:11:48,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41741,1689174599302' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-12 15:11:48,364 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-12 15:11:48,364 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-12 15:11:48,364 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-12 15:11:48,364 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-12 15:11:48,364 INFO [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-12 15:11:48,365 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,365 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,366 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,367 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-12 15:11:48,367 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-12 15:11:48,367 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-12 15:11:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,369 DEBUG [(jenkins-hbase20.apache.org,37639,1689174598377)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-12 15:11:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-12 15:11:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-12 15:11:48,369 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-12 15:11:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-12 15:11:48,370 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-12 15:11:48,470 DEBUG [Listener at localhost.localdomain/46471] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-12 15:11:48,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-12 15:11:48,478 DEBUG [Listener at localhost.localdomain/46471] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,479 DEBUG [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=c9e02faa319de90816e4c0f2fab9deb6, daughterA=dd42bee7259accb295dd58047b728cd2, daughterB=72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:48,480 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=c9e02faa319de90816e4c0f2fab9deb6, daughterA=dd42bee7259accb295dd58047b728cd2, daughterB=72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:48,480 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=c9e02faa319de90816e4c0f2fab9deb6, daughterA=dd42bee7259accb295dd58047b728cd2, daughterB=72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:48,480 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=c9e02faa319de90816e4c0f2fab9deb6, daughterA=dd42bee7259accb295dd58047b728cd2, daughterB=72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:48,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, UNASSIGN}] 2023-07-12 15:11:48,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, UNASSIGN 2023-07-12 15:11:48,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=c9e02faa319de90816e4c0f2fab9deb6, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:48,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174708483"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174708483"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174708483"}]},"ts":"1689174708483"} 2023-07-12 15:11:48,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure c9e02faa319de90816e4c0f2fab9deb6, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:48,635 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing c9e02faa319de90816e4c0f2fab9deb6, disabling compactions & flushes 2023-07-12 15:11:48,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. after waiting 0 ms 2023-07-12 15:11:48,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing c9e02faa319de90816e4c0f2fab9deb6 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-12 15:11:48,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/e341be2b802c42da840f10502a4bc663 2023-07-12 15:11:48,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/dbc7f79566b9406ab42c711e1d73380f 2023-07-12 15:11:48,657 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf1/e341be2b802c42da840f10502a4bc663 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663 2023-07-12 15:11:48,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:48,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/.tmp/cf2/dbc7f79566b9406ab42c711e1d73380f as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f 2023-07-12 15:11:48,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f, entries=8, sequenceid=82, filesize=5.0 K 2023-07-12 15:11:48,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c9e02faa319de90816e4c0f2fab9deb6 in 30ms, sequenceid=82, compaction requested=true 2023-07-12 15:11:48,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-12 15:11:48,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:48,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. 2023-07-12 15:11:48,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for c9e02faa319de90816e4c0f2fab9deb6: 2023-07-12 15:11:48,677 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,677 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=c9e02faa319de90816e4c0f2fab9deb6, regionState=CLOSED 2023-07-12 15:11:48,677 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174708677"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174708677"}]},"ts":"1689174708677"} 2023-07-12 15:11:48,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-07-12 15:11:48,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure c9e02faa319de90816e4c0f2fab9deb6, server=jenkins-hbase20.apache.org,41741,1689174599302 in 194 msec 2023-07-12 15:11:48,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-07-12 15:11:48,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c9e02faa319de90816e4c0f2fab9deb6, UNASSIGN in 197 msec 2023-07-12 15:11:48,688 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=c9e02faa319de90816e4c0f2fab9deb6, threads=14 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,689 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,690 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,690 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,690 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,838 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,839 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,871 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,871 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,874 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,883 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,885 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,885 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,903 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,904 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,905 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,905 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,906 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,906 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68 for region: c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:48,906 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region c9e02faa319de90816e4c0f2fab9deb6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-12 15:11:48,932 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:11:48,933 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-12 15:11:48,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1689174708934"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1689174708934"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1689174708934"}]},"ts":"1689174708934"} 2023-07-12 15:11:48,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174708934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174708934"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174708934"}]},"ts":"1689174708934"} 2023-07-12 15:11:48,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174708934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174708934"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174708934"}]},"ts":"1689174708934"} 2023-07-12 15:11:48,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, ASSIGN}] 2023-07-12 15:11:48,938 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, ASSIGN 2023-07-12 15:11:48,938 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:11:48,939 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, ASSIGN 2023-07-12 15:11:48,939 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41741,1689174599302; forceNewPlan=false, retain=false 2023-07-12 15:11:49,088 INFO [jenkins-hbase20:37639] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-12 15:11:49,090 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=72fb9eaa26c3750eb362ce01bfa490e4, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,090 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709090"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174709090"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174709090"}]},"ts":"1689174709090"} 2023-07-12 15:11:49,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 72fb9eaa26c3750eb362ce01bfa490e4, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:49,093 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=dd42bee7259accb295dd58047b728cd2, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,093 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709093"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174709093"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174709093"}]},"ts":"1689174709093"} 2023-07-12 15:11:49,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure dd42bee7259accb295dd58047b728cd2, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:49,179 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-12 15:11:49,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 72fb9eaa26c3750eb362ce01bfa490e4, NAME => 'testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.', STARTKEY => '41', ENDKEY => ''} 2023-07-12 15:11:49,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:49,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:49,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,247 INFO [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,248 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1 2023-07-12 15:11:49,248 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1 2023-07-12 15:11:49,248 INFO [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 72fb9eaa26c3750eb362ce01bfa490e4 columnFamilyName cf1 2023-07-12 15:11:49,258 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea-top 2023-07-12 15:11:49,261 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc-top 2023-07-12 15:11:49,264 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87-top 2023-07-12 15:11:49,266 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23-top 2023-07-12 15:11:49,269 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9-top 2023-07-12 15:11:49,271 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663-top 2023-07-12 15:11:49,274 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6-top 2023-07-12 15:11:49,274 INFO [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(310): Store=72fb9eaa26c3750eb362ce01bfa490e4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:49,274 INFO [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,275 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2 2023-07-12 15:11:49,275 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2 2023-07-12 15:11:49,275 INFO [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 72fb9eaa26c3750eb362ce01bfa490e4 columnFamilyName cf2 2023-07-12 15:11:49,285 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b-top 2023-07-12 15:11:49,287 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb-top 2023-07-12 15:11:49,290 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec-top 2023-07-12 15:11:49,293 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74-top 2023-07-12 15:11:49,296 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f-top 2023-07-12 15:11:49,299 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68-top 2023-07-12 15:11:49,302 DEBUG [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486-top 2023-07-12 15:11:49,302 INFO [StoreOpener-72fb9eaa26c3750eb362ce01bfa490e4-1] regionserver.HStore(310): Store=72fb9eaa26c3750eb362ce01bfa490e4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:49,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:49,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 72fb9eaa26c3750eb362ce01bfa490e4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72677566, jitterRate=0.08298012614250183}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:49,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 72fb9eaa26c3750eb362ce01bfa490e4: 2023-07-12 15:11:49,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4., pid=149, masterSystemTime=1689174709243 2023-07-12 15:11:49,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:49,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:11:49,307 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:49,307 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:49,307 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,307 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,307 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1912): 72fb9eaa26c3750eb362ce01bfa490e4/cf1 is initiating minor compaction (all files) 2023-07-12 15:11:49,307 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1912): 72fb9eaa26c3750eb362ce01bfa490e4/cf2 is initiating minor compaction (all files) 2023-07-12 15:11:49,307 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 72fb9eaa26c3750eb362ce01bfa490e4/cf1 in testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,307 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 72fb9eaa26c3750eb362ce01bfa490e4/cf2 in testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,308 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f-top] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/.tmp, totalSize=34.9 K 2023-07-12 15:11:49,308 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663-top] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/.tmp, totalSize=34.9 K 2023-07-12 15:11:49,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,308 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174707247 2023-07-12 15:11:49,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,308 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1689174707247 2023-07-12 15:11:49,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dd42bee7259accb295dd58047b728cd2, NAME => 'testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.', STARTKEY => '', ENDKEY => '41'} 2023-07-12 15:11:49,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=72fb9eaa26c3750eb362ce01bfa490e4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,309 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174707415 2023-07-12 15:11:49,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-12 15:11:49,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709309"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174709309"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174709309"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174709309"}]},"ts":"1689174709309"} 2023-07-12 15:11:49,309 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1689174707415 2023-07-12 15:11:49,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,309 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174707601 2023-07-12 15:11:49,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-12 15:11:49,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,309 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1689174707601 2023-07-12 15:11:49,309 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174707916 2023-07-12 15:11:49,310 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1689174707916 2023-07-12 15:11:49,310 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689174708104 2023-07-12 15:11:49,310 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1689174708104 2023-07-12 15:11:49,310 INFO [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,310 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689174708295 2023-07-12 15:11:49,310 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1689174708295 2023-07-12 15:11:49,310 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174708471 2023-07-12 15:11:49,311 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1689174708471 2023-07-12 15:11:49,311 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1 2023-07-12 15:11:49,311 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1 2023-07-12 15:11:49,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-07-12 15:11:49,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 72fb9eaa26c3750eb362ce01bfa490e4, server=jenkins-hbase20.apache.org,41741,1689174599302 in 219 msec 2023-07-12 15:11:49,311 INFO [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd42bee7259accb295dd58047b728cd2 columnFamilyName cf1 2023-07-12 15:11:49,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, ASSIGN in 374 msec 2023-07-12 15:11:49,319 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] throttle.PressureAwareThroughputController(145): 72fb9eaa26c3750eb362ce01bfa490e4#cf2#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:49,319 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] throttle.PressureAwareThroughputController(145): 72fb9eaa26c3750eb362ce01bfa490e4#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:49,323 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea-bottom 2023-07-12 15:11:49,336 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc-bottom 2023-07-12 15:11:49,338 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/.tmp/cf2/51c09a3533b84225bc74de931a7af12c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/51c09a3533b84225bc74de931a7af12c 2023-07-12 15:11:49,339 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/.tmp/cf1/65532bf392a94bf39be6dd0c1482bc49 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/65532bf392a94bf39be6dd0c1482bc49 2023-07-12 15:11:49,339 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87-bottom 2023-07-12 15:11:49,342 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23-bottom 2023-07-12 15:11:49,343 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 72fb9eaa26c3750eb362ce01bfa490e4/cf2 of 72fb9eaa26c3750eb362ce01bfa490e4 into 51c09a3533b84225bc74de931a7af12c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:49,343 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 72fb9eaa26c3750eb362ce01bfa490e4/cf1 of 72fb9eaa26c3750eb362ce01bfa490e4 into 65532bf392a94bf39be6dd0c1482bc49(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:49,343 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 72fb9eaa26c3750eb362ce01bfa490e4: 2023-07-12 15:11:49,343 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 72fb9eaa26c3750eb362ce01bfa490e4: 2023-07-12 15:11:49,343 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4., storeName=72fb9eaa26c3750eb362ce01bfa490e4/cf2, priority=9, startTime=1689174709307; duration=0sec 2023-07-12 15:11:49,343 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4., storeName=72fb9eaa26c3750eb362ce01bfa490e4/cf1, priority=9, startTime=1689174709306; duration=0sec 2023-07-12 15:11:49,343 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:49,343 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:49,348 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9-bottom 2023-07-12 15:11:49,350 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663-bottom 2023-07-12 15:11:49,353 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6-bottom 2023-07-12 15:11:49,353 INFO [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(310): Store=dd42bee7259accb295dd58047b728cd2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:49,353 INFO [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,354 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2 2023-07-12 15:11:49,354 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2 2023-07-12 15:11:49,354 INFO [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd42bee7259accb295dd58047b728cd2 columnFamilyName cf2 2023-07-12 15:11:49,364 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b-bottom 2023-07-12 15:11:49,367 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb-bottom 2023-07-12 15:11:49,369 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec-bottom 2023-07-12 15:11:49,372 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74-bottom 2023-07-12 15:11:49,374 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f-bottom 2023-07-12 15:11:49,377 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68-bottom 2023-07-12 15:11:49,379 DEBUG [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486-bottom 2023-07-12 15:11:49,380 INFO [StoreOpener-dd42bee7259accb295dd58047b728cd2-1] regionserver.HStore(310): Store=dd42bee7259accb295dd58047b728cd2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-12 15:11:49,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-12 15:11:49,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened dd42bee7259accb295dd58047b728cd2; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71748911, jitterRate=0.06914208829402924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-12 15:11:49,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for dd42bee7259accb295dd58047b728cd2: 2023-07-12 15:11:49,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2., pid=150, masterSystemTime=1689174709243 2023-07-12 15:11:49,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:49,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-12 15:11:49,384 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:49,384 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-12 15:11:49,385 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,385 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,385 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1912): dd42bee7259accb295dd58047b728cd2/cf2 is initiating minor compaction (all files) 2023-07-12 15:11:49,385 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1912): dd42bee7259accb295dd58047b728cd2/cf1 is initiating minor compaction (all files) 2023-07-12 15:11:49,385 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dd42bee7259accb295dd58047b728cd2/cf2 in testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,385 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2259): Starting compaction of dd42bee7259accb295dd58047b728cd2/cf1 in testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,385 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/.tmp, totalSize=34.9 K 2023-07-12 15:11:49,385 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f-bottom] into tmpdir=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/.tmp, totalSize=34.9 K 2023-07-12 15:11:49,386 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174707247 2023-07-12 15:11:49,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,386 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1689174707247 2023-07-12 15:11:49,386 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=dd42bee7259accb295dd58047b728cd2, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,386 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174707415 2023-07-12 15:11:49,386 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709386"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689174709386"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689174709386"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689174709386"}]},"ts":"1689174709386"} 2023-07-12 15:11:49,386 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1689174707415 2023-07-12 15:11:49,386 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174707601 2023-07-12 15:11:49,386 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1689174707601 2023-07-12 15:11:49,387 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174707916 2023-07-12 15:11:49,387 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting 44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1689174707916 2023-07-12 15:11:49,387 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting 2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174708104 2023-07-12 15:11:49,387 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1689174708104 2023-07-12 15:11:49,388 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174708295 2023-07-12 15:11:49,388 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1689174708295 2023-07-12 15:11:49,388 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] compactions.Compactor(207): Compacting e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174708471 2023-07-12 15:11:49,388 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] compactions.Compactor(207): Compacting dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1689174708471 2023-07-12 15:11:49,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-07-12 15:11:49,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure dd42bee7259accb295dd58047b728cd2, server=jenkins-hbase20.apache.org,41741,1689174599302 in 293 msec 2023-07-12 15:11:49,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-07-12 15:11:49,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, ASSIGN in 451 msec 2023-07-12 15:11:49,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=c9e02faa319de90816e4c0f2fab9deb6, daughterA=dd42bee7259accb295dd58047b728cd2, daughterB=72fb9eaa26c3750eb362ce01bfa490e4 in 912 msec 2023-07-12 15:11:49,395 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] throttle.PressureAwareThroughputController(145): dd42bee7259accb295dd58047b728cd2#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:49,395 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] throttle.PressureAwareThroughputController(145): dd42bee7259accb295dd58047b728cd2#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-12 15:11:49,403 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/.tmp/cf2/75a64454867e4c038f8e63dc3f1b0f71 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75a64454867e4c038f8e63dc3f1b0f71 2023-07-12 15:11:49,403 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/.tmp/cf1/77bc4d34f89545a1b29149bc1cf8cf9a as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/77bc4d34f89545a1b29149bc1cf8cf9a 2023-07-12 15:11:49,406 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dd42bee7259accb295dd58047b728cd2/cf1 of dd42bee7259accb295dd58047b728cd2 into 77bc4d34f89545a1b29149bc1cf8cf9a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:49,407 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd42bee7259accb295dd58047b728cd2: 2023-07-12 15:11:49,407 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dd42bee7259accb295dd58047b728cd2/cf2 of dd42bee7259accb295dd58047b728cd2 into 75a64454867e4c038f8e63dc3f1b0f71(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-12 15:11:49,407 INFO [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2., storeName=dd42bee7259accb295dd58047b728cd2/cf1, priority=9, startTime=1689174709384; duration=0sec 2023-07-12 15:11:49,407 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd42bee7259accb295dd58047b728cd2: 2023-07-12 15:11:49,407 INFO [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2., storeName=dd42bee7259accb295dd58047b728cd2/cf2, priority=9, startTime=1689174709384; duration=0sec 2023-07-12 15:11:49,407 DEBUG [RS:0;jenkins-hbase20:41741-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:49,407 DEBUG [RS:0;jenkins-hbase20:41741-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-12 15:11:49,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41741] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:35362 deadline: 1689174769481, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6. is not online on jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,590 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:11:49,598 INFO [Listener at localhost.localdomain/46471] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-12 15:11:49,599 DEBUG [Listener at localhost.localdomain/46471] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-12 15:11:49,611 WARN [Listener at localhost.localdomain/46471] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-12 15:11:49,613 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-07-12 15:11:49,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegion 2023-07-12 15:11:49,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-12 15:11:49,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174709616"}]},"ts":"1689174709616"} 2023-07-12 15:11:49,617 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-07-12 15:11:49,618 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-07-12 15:11:49,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, UNASSIGN}] 2023-07-12 15:11:49,619 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, UNASSIGN 2023-07-12 15:11:49,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=72fb9eaa26c3750eb362ce01bfa490e4, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,620 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709620"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174709620"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174709620"}]},"ts":"1689174709620"} 2023-07-12 15:11:49,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 72fb9eaa26c3750eb362ce01bfa490e4, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:49,622 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, UNASSIGN 2023-07-12 15:11:49,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=dd42bee7259accb295dd58047b728cd2, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:49,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709623"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689174709623"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689174709623"}]},"ts":"1689174709623"} 2023-07-12 15:11:49,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure dd42bee7259accb295dd58047b728cd2, server=jenkins-hbase20.apache.org,41741,1689174599302}] 2023-07-12 15:11:49,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-12 15:11:49,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing dd42bee7259accb295dd58047b728cd2, disabling compactions & flushes 2023-07-12 15:11:49,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. after waiting 0 ms 2023-07-12 15:11:49,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,773 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663-bottom] to archive 2023-07-12 15:11:49,774 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:49,775 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,776 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,777 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,777 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,778 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,779 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,780 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,781 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486-bottom, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f-bottom] to archive 2023-07-12 15:11:49,781 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:49,782 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,783 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,784 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,785 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,785 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,786 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,787 DEBUG [StoreCloser-testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:11:49,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:49,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2. 2023-07-12 15:11:49,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for dd42bee7259accb295dd58047b728cd2: 2023-07-12 15:11:49,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(111): Close 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 72fb9eaa26c3750eb362ce01bfa490e4, disabling compactions & flushes 2023-07-12 15:11:49,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. after waiting 0 ms 2023-07-12 15:11:49,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,792 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=dd42bee7259accb295dd58047b728cd2, regionState=CLOSED 2023-07-12 15:11:49,792 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709792"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174709792"}]},"ts":"1689174709792"} 2023-07-12 15:11:49,793 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663-top] to archive 2023-07-12 15:11:49,793 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:49,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-07-12 15:11:49,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure dd42bee7259accb295dd58047b728cd2, server=jenkins-hbase20.apache.org,41741,1689174599302 in 169 msec 2023-07-12 15:11:49,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=dd42bee7259accb295dd58047b728cd2, UNASSIGN in 175 msec 2023-07-12 15:11:49,795 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/f041fd77d7194f809eb8ec23a62c32d6.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,795 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/cb3e3368115142f78fcdfcbcae869e23.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,796 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e0208a161cc34887baf5463e3cde00a9.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,797 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b66a4b3dbdc84966905ec1db44042e87.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,797 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/2f4846d1be544d7392b5c386c4a905ea.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,798 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,799 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/e341be2b802c42da840f10502a4bc663.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,800 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486-top, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6->hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f-top] to archive 2023-07-12 15:11:49,801 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-12 15:11:49,802 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/75e58a986e30408ca0390d3f68603b74.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,803 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/4ad5389de4614580a29418a35865abec.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,804 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/0dd0d393a0944bc5904f0d0195d07a4b.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,804 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/44f2a36ef5484441b8c2b40bd8cb85eb.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,805 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/ed61459025724feeae7259c340597e68.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,806 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/f361910f5fbc418a9a2135f1baead486.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,806 DEBUG [StoreCloser-testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/dbc7f79566b9406ab42c711e1d73380f.c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-12 15:11:49,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:49,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4. 2023-07-12 15:11:49,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 72fb9eaa26c3750eb362ce01bfa490e4: 2023-07-12 15:11:49,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(149): Closed 72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,811 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=72fb9eaa26c3750eb362ce01bfa490e4, regionState=CLOSED 2023-07-12 15:11:49,811 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1689174709811"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689174709811"}]},"ts":"1689174709811"} 2023-07-12 15:11:49,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-07-12 15:11:49,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 72fb9eaa26c3750eb362ce01bfa490e4, server=jenkins-hbase20.apache.org,41741,1689174599302 in 190 msec 2023-07-12 15:11:49,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-07-12 15:11:49,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=72fb9eaa26c3750eb362ce01bfa490e4, UNASSIGN in 193 msec 2023-07-12 15:11:49,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689174709814"}]},"ts":"1689174709814"} 2023-07-12 15:11:49,814 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-07-12 15:11:49,815 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-07-12 15:11:49,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 202 msec 2023-07-12 15:11:49,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-12 15:11:49,918 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-07-12 15:11:49,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegion 2023-07-12 15:11:49,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,921 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,921 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-12 15:11:49,925 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,925 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,925 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,926 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/recovered.edits] 2023-07-12 15:11:49,926 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/recovered.edits] 2023-07-12 15:11:49,926 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2, FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/recovered.edits] 2023-07-12 15:11:49,928 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/77bc4d34f89545a1b29149bc1cf8cf9a to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf1/77bc4d34f89545a1b29149bc1cf8cf9a 2023-07-12 15:11:49,928 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/65532bf392a94bf39be6dd0c1482bc49 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf1/65532bf392a94bf39be6dd0c1482bc49 2023-07-12 15:11:49,929 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/2f4846d1be544d7392b5c386c4a905ea 2023-07-12 15:11:49,930 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b4e8d1798b5d45e9b05e45b21f27e4dc 2023-07-12 15:11:49,930 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75a64454867e4c038f8e63dc3f1b0f71 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/cf2/75a64454867e4c038f8e63dc3f1b0f71 2023-07-12 15:11:49,930 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/51c09a3533b84225bc74de931a7af12c to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/cf2/51c09a3533b84225bc74de931a7af12c 2023-07-12 15:11:49,931 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/b66a4b3dbdc84966905ec1db44042e87 2023-07-12 15:11:49,932 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/cb3e3368115142f78fcdfcbcae869e23 2023-07-12 15:11:49,932 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4/recovered.edits/90.seqid 2023-07-12 15:11:49,932 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/recovered.edits/90.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2/recovered.edits/90.seqid 2023-07-12 15:11:49,933 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/dd42bee7259accb295dd58047b728cd2 2023-07-12 15:11:49,933 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/72fb9eaa26c3750eb362ce01bfa490e4 2023-07-12 15:11:49,933 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e0208a161cc34887baf5463e3cde00a9 2023-07-12 15:11:49,934 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/e341be2b802c42da840f10502a4bc663 2023-07-12 15:11:49,935 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf1/f041fd77d7194f809eb8ec23a62c32d6 2023-07-12 15:11:49,937 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/0dd0d393a0944bc5904f0d0195d07a4b 2023-07-12 15:11:49,938 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/44f2a36ef5484441b8c2b40bd8cb85eb 2023-07-12 15:11:49,939 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/4ad5389de4614580a29418a35865abec 2023-07-12 15:11:49,940 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/75e58a986e30408ca0390d3f68603b74 2023-07-12 15:11:49,941 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/dbc7f79566b9406ab42c711e1d73380f 2023-07-12 15:11:49,942 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/ed61459025724feeae7259c340597e68 2023-07-12 15:11:49,942 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486 to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/cf2/f361910f5fbc418a9a2135f1baead486 2023-07-12 15:11:49,944 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/recovered.edits/85.seqid to hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/archive/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6/recovered.edits/85.seqid 2023-07-12 15:11:49,945 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/.tmp/data/default/testSplitTableRegion/c9e02faa319de90816e4c0f2fab9deb6 2023-07-12 15:11:49,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-12 15:11:49,946 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,947 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-07-12 15:11:49,949 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-07-12 15:11:49,949 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-07-12 15:11:49,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174709950"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:49,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174709950"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:49,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1689174709950"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:49,951 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-12 15:11:49,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c9e02faa319de90816e4c0f2fab9deb6, NAME => 'testSplitTableRegion,,1689174706743.c9e02faa319de90816e4c0f2fab9deb6.', STARTKEY => '', ENDKEY => ''}, {ENCODED => dd42bee7259accb295dd58047b728cd2, NAME => 'testSplitTableRegion,,1689174708478.dd42bee7259accb295dd58047b728cd2.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 72fb9eaa26c3750eb362ce01bfa490e4, NAME => 'testSplitTableRegion,41,1689174708478.72fb9eaa26c3750eb362ce01bfa490e4.', STARTKEY => '41', ENDKEY => ''}] 2023-07-12 15:11:49,951 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-07-12 15:11:49,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1689174709951"}]},"ts":"9223372036854775807"} 2023-07-12 15:11:49,952 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-07-12 15:11:49,953 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-12 15:11:49,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 34 msec 2023-07-12 15:11:50,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37639] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-12 15:11:50,023 INFO [Listener at localhost.localdomain/46471] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-07-12 15:11:50,034 INFO [Listener at localhost.localdomain/46471] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379 (was 376) - Thread LEAK? -, OpenFileDescriptor=829 (was 842), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=113 (was 98) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=5227 (was 5305) 2023-07-12 15:11:50,034 INFO [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-12 15:11:50,037 INFO [Listener at localhost.localdomain/46471] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-12 15:11:50,038 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x722c7441 to 127.0.0.1:63723 2023-07-12 15:11:50,038 DEBUG [Listener at localhost.localdomain/46471] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,039 DEBUG [Listener at localhost.localdomain/46471] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-12 15:11:50,040 DEBUG [Listener at localhost.localdomain/46471] util.JVMClusterUtil(257): Found active master hash=1274185319, stopped=false 2023-07-12 15:11:50,040 INFO [Listener at localhost.localdomain/46471] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 15:11:50,042 INFO [Listener at localhost.localdomain/46471] procedure2.ProcedureExecutor(629): Stopping 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-12 15:11:50,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:11:50,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5fb4a008 to 127.0.0.1:63723 2023-07-12 15:11:50,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:11:50,042 DEBUG [Listener at localhost.localdomain/46471] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-12 15:11:50,043 INFO [Listener at localhost.localdomain/46471] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,41741,1689174599302' ***** 2023-07-12 15:11:50,043 INFO [Listener at localhost.localdomain/46471] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:11:50,043 INFO [Listener at localhost.localdomain/46471] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,33703,1689174599345' ***** 2023-07-12 15:11:50,043 INFO [Listener at localhost.localdomain/46471] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:11:50,043 INFO [Listener at localhost.localdomain/46471] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,37993,1689174599377' ***** 2023-07-12 15:11:50,043 INFO [Listener at localhost.localdomain/46471] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-12 15:11:50,043 INFO [RS:1;jenkins-hbase20:33703] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:11:50,043 INFO [RS:2;jenkins-hbase20:37993] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:11:50,043 INFO [RS:0;jenkins-hbase20:41741] regionserver.HeapMemoryManager(220): Stopping 2023-07-12 15:11:50,043 INFO [RS:1;jenkins-hbase20:33703] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:11:50,043 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:11:50,043 INFO [RS:1;jenkins-hbase20:33703] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:11:50,043 INFO [RS:0;jenkins-hbase20:41741] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:11:50,043 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:50,043 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:11:50,044 DEBUG [RS:1;jenkins-hbase20:33703] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x41445b67 to 127.0.0.1:63723 2023-07-12 15:11:50,043 INFO [RS:2;jenkins-hbase20:37993] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-12 15:11:50,043 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-12 15:11:50,044 INFO [RS:2;jenkins-hbase20:37993] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:11:50,044 DEBUG [RS:1;jenkins-hbase20:33703] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,043 INFO [RS:0;jenkins-hbase20:41741] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-12 15:11:50,044 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(3305): Received CLOSE for 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:11:50,044 INFO [RS:1;jenkins-hbase20:33703] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:11:50,044 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:50,044 INFO [RS:1;jenkins-hbase20:33703] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:11:50,044 DEBUG [RS:0;jenkins-hbase20:41741] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c27caab to 127.0.0.1:63723 2023-07-12 15:11:50,044 INFO [RS:1;jenkins-hbase20:33703] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:11:50,044 DEBUG [RS:0;jenkins-hbase20:41741] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,044 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-12 15:11:50,044 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:50,044 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,41741,1689174599302; all regions closed. 2023-07-12 15:11:50,044 DEBUG [RS:2;jenkins-hbase20:37993] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d90a6b4 to 127.0.0.1:63723 2023-07-12 15:11:50,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 688006d0ea2ee3fc16a6d36bae9436b7, disabling compactions & flushes 2023-07-12 15:11:50,044 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-12 15:11:50,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:11:50,044 DEBUG [RS:2;jenkins-hbase20:37993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,044 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-12 15:11:50,044 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:11:50,044 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-12 15:11:50,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. after waiting 0 ms 2023-07-12 15:11:50,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:11:50,044 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-12 15:11:50,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 688006d0ea2ee3fc16a6d36bae9436b7 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-12 15:11:50,045 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-12 15:11:50,045 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1478): Online Regions={688006d0ea2ee3fc16a6d36bae9436b7=hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7.} 2023-07-12 15:11:50,045 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-12 15:11:50,045 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-12 15:11:50,045 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-12 15:11:50,045 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-07-12 15:11:50,045 DEBUG [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-12 15:11:50,045 DEBUG [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1504): Waiting on 688006d0ea2ee3fc16a6d36bae9436b7 2023-07-12 15:11:50,055 DEBUG [RS:0;jenkins-hbase20:41741] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs 2023-07-12 15:11:50,055 INFO [RS:0;jenkins-hbase20:41741] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C41741%2C1689174599302:(num 1689174600728) 2023-07-12 15:11:50,055 DEBUG [RS:0;jenkins-hbase20:41741] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,055 INFO [RS:0;jenkins-hbase20:41741] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:11:50,056 INFO [RS:0;jenkins-hbase20:41741] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:11:50,056 INFO [RS:0;jenkins-hbase20:41741] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:11:50,056 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:11:50,056 INFO [RS:0;jenkins-hbase20:41741] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:11:50,056 INFO [RS:0;jenkins-hbase20:41741] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:11:50,062 INFO [RS:0;jenkins-hbase20:41741] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:41741 2023-07-12 15:11:50,066 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:11:50,066 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:11:50,066 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:11:50,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/.tmp/info/b0b78db749234024bcae8af0813b8700 2023-07-12 15:11:50,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:50,067 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:11:50,067 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:11:50,066 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:50,067 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41741,1689174599302 2023-07-12 15:11:50,067 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:11:50,067 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:11:50,067 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,41741,1689174599302] 2023-07-12 15:11:50,067 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,41741,1689174599302; numProcessing=1 2023-07-12 15:11:50,068 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,41741,1689174599302 already deleted, retry=false 2023-07-12 15:11:50,068 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,41741,1689174599302 expired; onlineServers=2 2023-07-12 15:11:50,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/.tmp/info/b0b78db749234024bcae8af0813b8700 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/info/b0b78db749234024bcae8af0813b8700 2023-07-12 15:11:50,072 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/.tmp/info/221b6e5405f647e28d16882faed92f5c 2023-07-12 15:11:50,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/info/b0b78db749234024bcae8af0813b8700, entries=2, sequenceid=6, filesize=4.8 K 2023-07-12 15:11:50,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 688006d0ea2ee3fc16a6d36bae9436b7 in 30ms, sequenceid=6, compaction requested=false 2023-07-12 15:11:50,076 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 221b6e5405f647e28d16882faed92f5c 2023-07-12 15:11:50,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/namespace/688006d0ea2ee3fc16a6d36bae9436b7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-12 15:11:50,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:50,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:11:50,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 688006d0ea2ee3fc16a6d36bae9436b7: 2023-07-12 15:11:50,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689174601198.688006d0ea2ee3fc16a6d36bae9436b7. 2023-07-12 15:11:50,088 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/.tmp/rep_barrier/b7ed887213e54b69849c5736d89c5c71 2023-07-12 15:11:50,091 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7ed887213e54b69849c5736d89c5c71 2023-07-12 15:11:50,103 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/.tmp/table/9a009bf676fe4f01b4139ad642f8220c 2023-07-12 15:11:50,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9a009bf676fe4f01b4139ad642f8220c 2023-07-12 15:11:50,106 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/.tmp/info/221b6e5405f647e28d16882faed92f5c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/info/221b6e5405f647e28d16882faed92f5c 2023-07-12 15:11:50,109 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 221b6e5405f647e28d16882faed92f5c 2023-07-12 15:11:50,109 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/info/221b6e5405f647e28d16882faed92f5c, entries=59, sequenceid=210, filesize=11.1 K 2023-07-12 15:11:50,109 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/.tmp/rep_barrier/b7ed887213e54b69849c5736d89c5c71 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/rep_barrier/b7ed887213e54b69849c5736d89c5c71 2023-07-12 15:11:50,112 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7ed887213e54b69849c5736d89c5c71 2023-07-12 15:11:50,112 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/rep_barrier/b7ed887213e54b69849c5736d89c5c71, entries=24, sequenceid=210, filesize=7.5 K 2023-07-12 15:11:50,112 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/.tmp/table/9a009bf676fe4f01b4139ad642f8220c as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/table/9a009bf676fe4f01b4139ad642f8220c 2023-07-12 15:11:50,115 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9a009bf676fe4f01b4139ad642f8220c 2023-07-12 15:11:50,115 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/table/9a009bf676fe4f01b4139ad642f8220c, entries=36, sequenceid=210, filesize=8.0 K 2023-07-12 15:11:50,116 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 71ms, sequenceid=210, compaction requested=false 2023-07-12 15:11:50,119 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-07-12 15:11:50,119 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-12 15:11:50,119 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-12 15:11:50,120 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-12 15:11:50,120 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-12 15:11:50,120 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-12 15:11:50,168 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,168 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:41741-0x10081bb5c6c0001, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,168 INFO [RS:0;jenkins-hbase20:41741] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,41741,1689174599302; zookeeper connection closed. 2023-07-12 15:11:50,168 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7fed7f07] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7fed7f07 2023-07-12 15:11:50,245 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,33703,1689174599345; all regions closed. 2023-07-12 15:11:50,245 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,37993,1689174599377; all regions closed. 2023-07-12 15:11:50,252 DEBUG [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs 2023-07-12 15:11:50,252 DEBUG [RS:2;jenkins-hbase20:37993] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs 2023-07-12 15:11:50,252 INFO [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C33703%2C1689174599345.meta:.meta(num 1689174600946) 2023-07-12 15:11:50,252 INFO [RS:2;jenkins-hbase20:37993] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37993%2C1689174599377:(num 1689174600727) 2023-07-12 15:11:50,252 DEBUG [RS:2;jenkins-hbase20:37993] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,252 INFO [RS:2;jenkins-hbase20:37993] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:11:50,254 INFO [RS:2;jenkins-hbase20:37993] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-12 15:11:50,254 INFO [RS:2;jenkins-hbase20:37993] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-12 15:11:50,254 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:11:50,254 INFO [RS:2;jenkins-hbase20:37993] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-12 15:11:50,255 INFO [RS:2;jenkins-hbase20:37993] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-12 15:11:50,255 INFO [RS:2;jenkins-hbase20:37993] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:37993 2023-07-12 15:11:50,258 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:50,258 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:11:50,258 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37993,1689174599377 2023-07-12 15:11:50,260 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,37993,1689174599377] 2023-07-12 15:11:50,260 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,37993,1689174599377; numProcessing=2 2023-07-12 15:11:50,260 DEBUG [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/oldWALs 2023-07-12 15:11:50,260 INFO [RS:1;jenkins-hbase20:33703] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C33703%2C1689174599345:(num 1689174600727) 2023-07-12 15:11:50,260 DEBUG [RS:1;jenkins-hbase20:33703] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,261 INFO [RS:1;jenkins-hbase20:33703] regionserver.LeaseManager(133): Closed leases 2023-07-12 15:11:50,261 INFO [RS:1;jenkins-hbase20:33703] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-12 15:11:50,261 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:11:50,261 INFO [RS:1;jenkins-hbase20:33703] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:33703 2023-07-12 15:11:50,360 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,360 INFO [RS:2;jenkins-hbase20:37993] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,37993,1689174599377; zookeeper connection closed. 2023-07-12 15:11:50,360 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:37993-0x10081bb5c6c0003, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,360 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@72cc37da] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@72cc37da 2023-07-12 15:11:50,361 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,37993,1689174599377 already deleted, retry=false 2023-07-12 15:11:50,361 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-12 15:11:50,361 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,37993,1689174599377 expired; onlineServers=1 2023-07-12 15:11:50,361 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,33703,1689174599345 2023-07-12 15:11:50,361 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,33703,1689174599345] 2023-07-12 15:11:50,361 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,33703,1689174599345; numProcessing=3 2023-07-12 15:11:50,362 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,33703,1689174599345 already deleted, retry=false 2023-07-12 15:11:50,362 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,33703,1689174599345 expired; onlineServers=0 2023-07-12 15:11:50,362 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,37639,1689174598377' ***** 2023-07-12 15:11:50,362 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-12 15:11:50,363 DEBUG [M:0;jenkins-hbase20:37639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@acf8c5c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-12 15:11:50,363 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,37639,1689174598377 2023-07-12 15:11:50,363 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,37639,1689174598377; all regions closed. 2023-07-12 15:11:50,363 DEBUG [M:0;jenkins-hbase20:37639] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-12 15:11:50,363 DEBUG [M:0;jenkins-hbase20:37639] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-12 15:11:50,363 DEBUG [M:0;jenkins-hbase20:37639] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-12 15:11:50,363 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689174600394] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689174600394,5,FailOnTimeoutGroup] 2023-07-12 15:11:50,363 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-12 15:11:50,363 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689174600394] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689174600394,5,FailOnTimeoutGroup] 2023-07-12 15:11:50,364 INFO [M:0;jenkins-hbase20:37639] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-12 15:11:50,364 INFO [M:0;jenkins-hbase20:37639] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-12 15:11:50,364 INFO [M:0;jenkins-hbase20:37639] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-07-12 15:11:50,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-12 15:11:50,364 DEBUG [M:0;jenkins-hbase20:37639] master.HMaster(1512): Stopping service threads 2023-07-12 15:11:50,364 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-12 15:11:50,364 INFO [M:0;jenkins-hbase20:37639] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-12 15:11:50,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-12 15:11:50,364 ERROR [M:0;jenkins-hbase20:37639] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-07-12 15:11:50,364 INFO [M:0;jenkins-hbase20:37639] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-12 15:11:50,364 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-12 15:11:50,365 DEBUG [M:0;jenkins-hbase20:37639] zookeeper.ZKUtil(398): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-12 15:11:50,365 WARN [M:0;jenkins-hbase20:37639] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-12 15:11:50,365 INFO [M:0;jenkins-hbase20:37639] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-12 15:11:50,365 INFO [M:0;jenkins-hbase20:37639] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-12 15:11:50,365 DEBUG [M:0;jenkins-hbase20:37639] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-12 15:11:50,365 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:11:50,365 DEBUG [M:0;jenkins-hbase20:37639] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:11:50,365 DEBUG [M:0;jenkins-hbase20:37639] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-12 15:11:50,365 DEBUG [M:0;jenkins-hbase20:37639] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:11:50,365 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.83 KB heapSize=719.72 KB 2023-07-12 15:11:50,380 INFO [M:0;jenkins-hbase20:37639] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.83 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b7139cf4b5e4426fb1baac2533041870 2023-07-12 15:11:50,383 ERROR [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid 631c6a2a-c60e-45ea-8ae8-e13efaf3ce20) service to localhost.localdomain/127.0.0.1:34607 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-12 15:11:50,384 INFO [M:0;jenkins-hbase20:37639] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7139cf4b5e4426fb1baac2533041870 2023-07-12 15:11:50,384 DEBUG [M:0;jenkins-hbase20:37639] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b7139cf4b5e4426fb1baac2533041870 as hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b7139cf4b5e4426fb1baac2533041870 2023-07-12 15:11:50,388 INFO [M:0;jenkins-hbase20:37639] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7139cf4b5e4426fb1baac2533041870 2023-07-12 15:11:50,388 INFO [M:0;jenkins-hbase20:37639] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34607/user/jenkins/test-data/35335602-46e9-8380-2e45-861116761fe7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b7139cf4b5e4426fb1baac2533041870, entries=156, sequenceid=1282, filesize=34.9 K 2023-07-12 15:11:50,389 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegion(2948): Finished flush of dataSize ~608.83 KB/623438, heapSize ~719.70 KB/736976, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 23ms, sequenceid=1282, compaction requested=false 2023-07-12 15:11:50,389 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-12 15:11:50,389 DEBUG [M:0;jenkins-hbase20:37639] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-12 15:11:50,392 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-12 15:11:50,392 INFO [M:0;jenkins-hbase20:37639] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-12 15:11:50,393 INFO [M:0;jenkins-hbase20:37639] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:37639 2023-07-12 15:11:50,394 DEBUG [M:0;jenkins-hbase20:37639] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,37639,1689174598377 already deleted, retry=false 2023-07-12 15:11:50,461 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,461 INFO [RS:1;jenkins-hbase20:33703] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,33703,1689174599345; zookeeper connection closed. 2023-07-12 15:11:50,462 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): regionserver:33703-0x10081bb5c6c0002, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,462 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@ac97765] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@ac97765 2023-07-12 15:11:50,462 INFO [Listener at localhost.localdomain/46471] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-07-12 15:11:50,562 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,562 DEBUG [Listener at localhost.localdomain/46471-EventThread] zookeeper.ZKWatcher(600): master:37639-0x10081bb5c6c0000, quorum=127.0.0.1:63723, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-12 15:11:50,562 INFO [M:0;jenkins-hbase20:37639] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,37639,1689174598377; zookeeper connection closed. 2023-07-12 15:11:50,564 WARN [Listener at localhost.localdomain/46471] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:11:50,567 INFO [Listener at localhost.localdomain/46471] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:11:50,679 WARN [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid 631c6a2a-c60e-45ea-8ae8-e13efaf3ce20) service to localhost.localdomain/127.0.0.1:34607 2023-07-12 15:11:50,681 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data5/current/BP-1364318182-148.251.75.209-1689174595739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:11:50,681 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data6/current/BP-1364318182-148.251.75.209-1689174595739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:11:50,683 WARN [Listener at localhost.localdomain/46471] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:11:50,684 INFO [Listener at localhost.localdomain/46471] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:11:50,793 WARN [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-12 15:11:50,793 WARN [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid d6c01eab-2c69-419b-b4a6-953858ab157f) service to localhost.localdomain/127.0.0.1:34607 2023-07-12 15:11:50,794 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data3/current/BP-1364318182-148.251.75.209-1689174595739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:11:50,795 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data4/current/BP-1364318182-148.251.75.209-1689174595739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:11:50,796 WARN [Listener at localhost.localdomain/46471] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-12 15:11:50,798 INFO [Listener at localhost.localdomain/46471] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-12 15:11:50,900 WARN [BP-1364318182-148.251.75.209-1689174595739 heartbeating to localhost.localdomain/127.0.0.1:34607] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1364318182-148.251.75.209-1689174595739 (Datanode Uuid 5606ee4f-c045-409e-a742-091e025e583c) service to localhost.localdomain/127.0.0.1:34607 2023-07-12 15:11:50,902 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data1/current/BP-1364318182-148.251.75.209-1689174595739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:11:50,902 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/759bfb5f-f942-7527-4f1b-aa737d620cc2/cluster_25af260d-c5b4-24f4-d76c-531da32eaea7/dfs/data/data2/current/BP-1364318182-148.251.75.209-1689174595739] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-12 15:11:50,925 INFO [Listener at localhost.localdomain/46471] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-12 15:11:51,106 INFO [Listener at localhost.localdomain/46471] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-12 15:11:51,324 INFO [Listener at localhost.localdomain/46471] hbase.HBaseTestingUtility(1293): Minicluster is down